Introduction
In this series I will be going through how to build an HTTP Server from scratch in C++ using POSIX APIs, and actually have it serve a web page which has CRUD operations to your browser - also it would be capable of serving more than one client at the same time!
Note: I am using C++ 17. And I will be dividing this whole project into multiple parts (maybe 4-5 blogs for now).
What can you learn from this project?
Well for starters, you will get to know how TCP sockets work and how to use them to build a server. You will also get to know how HTTP works and how to parse it. You will get to know how to build a web page and how to serve it to the browser. And finally, you will get to understand how complex this architecture gets when trying to serve multiple clients and getting millions of requests! Although I won't be making a server which can handle millions of requests, hopefully you can appreciate the actual server serving you this blog post.
Without further ado, let's start with our first part of this series: TCP Sockets!
No more yapping
Use these includes:
#include <iostream>
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#include <arpa/inet.h>
#include <string>
Enter fullscreen mode Exit fullscreen mode
Creating a socket
For the server to accept requests and reply with responses, we need a medium where the server and client can be connected (talk to each other). For that we use sockets. A socket can be thought of as an endpoint of communication between two programs running on a network.
int server_fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (server_fd < 0)
{
std::cerr << "error creating the socket" << std::endl;
return -1;
}
Enter fullscreen mode Exit fullscreen mode
The above code creates a socket. AF_INET means we are using IPv4. SOCK_STREAM means we are using TCP. IPPROTO_TCP means we are using the TCP protocol. Alternatively you can just pass 0 as the last argument and the kernel will figure out the protocol based on the socket type (in this case TCP because we used SOCK_STREAM).
The socket function returns an integer file descriptor, which we can use to refer to this socket. Think of a file descriptor as a unique identifier for this socket. If the file descriptor is -1, this means it failed to create a socket.
I recommend adding this snippet after creating a socket:
int opt{1};
setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
Enter fullscreen mode Exit fullscreen mode
When you are testing your code you will probably need to start your server multiple times (just like I did). However if you don't do the above step, then you will get an error which tells you that the address is already in use. This happens because the socket gets stuck in TIME_WAIT state after the server is closed. The above code prevents this and lets you reuse the same address immediately after the server is closed.
Binding the socket to an address
Now we have the socket, but it is not connected to any address. So we need to bind it to an address. The address is a combination of an IP address and a port number.
struct sockaddr_in server_addr{};
server_addr.sin_addr.s_addr = INADDR_ANY;
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(8080);
if (bind(server_fd, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0)
{
std::cerr << "error binding to the socket" << std::endl;
return -1;
}
Enter fullscreen mode Exit fullscreen mode
Here server_addr is a struct which holds the address of the server. sin_addr.s_addr = INADDR_ANY means that the server will listen on all available interfaces. sin_family = AF_INET means we are using IPv4. sin_port = htons(8080) means we are using port 8080. htons() is used to convert the port number to network byte order.
We bind the server using the bind() function, which takes the socket file descriptor, the address of the server, and the size of the address as arguments. Again, if this returns -1, it means it failed to bind the socket.
Listening for incoming connections
Now the server is bound to a certain address, meaning it has a home now. But it is still closed and doesn't accept visitors, so we need to make it listen for incoming connections.
if (listen(server_fd, 10) < 0)
{
std::cerr << "error listening on the socket" << std::endl;
return -1;
}
Enter fullscreen mode Exit fullscreen mode
The listen() function tells the kernel that the socket is a passive socket, and it should be used to accept incoming connections. The second argument 10 is the maximum number of pending connections that can be queued. If the queue is full, incoming connections will be rejected. If this returns -1, it means it failed to listen on the socket.
Putting it all together
Now that we have our server ready, we can start accepting connections. But before that, I will put all the above code in a function named startServer():
int startServer()
{
int server_fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (server_fd < 0)
{
std::cerr << "error creating the socket" << std::endl;
return -1;
}
int opt{1};
setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
struct sockaddr_in server_addr{};
server_addr.sin_addr.s_addr = INADDR_ANY;
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(8080);
if (bind(server_fd, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0)
{
std::cerr << "error binding to the socket" << std::endl;
return -1;
}
if (listen(server_fd, 10) < 0)
{
std::cerr << "error listening on the socket" << std::endl;
return -1;
}
return server_fd;
}
Enter fullscreen mode Exit fullscreen mode
Accepting incoming connections
Let's call this function getClient()
int getClient(int server_fd)
{
struct sockaddr_in client_addr{};
socklen_t client_len = sizeof(client_addr);
int client_fd = accept(server_fd, (struct sockaddr *)&client_addr, &client_len);
if (client_fd < 0)
{
std::cerr << "error accepting the client" << std::endl;
return -1;
}
std::cout << "client connected to socket successfully! \n";
return client_fd;
}
Enter fullscreen mode Exit fullscreen mode
The above code now accepts the connection by calling accept() on the server file descriptor, and stores the client address in the client_addr struct. It returns the client file descriptor if successful, and -1 if it fails. client_fd can be understood as the unique identifier for the client's socket.
Communication between Client and Server
Since our server is running and accepting clients, next we need to accept their requests and provide appropriate responses. But to do that, we need to receive data from the client and send data to the client.
Receiving data from Client
char buffer[8193]{};
int bytes = recv(client_fd, buffer, sizeof(buffer) - 1, 0);
buffer[bytes] = '\0';
Enter fullscreen mode Exit fullscreen mode
We use the recv() function to receive data from the client. It needs the client file descriptor to identify which client we are talking to, a buffer to store the data, the size of the buffer, and finally a flag set to 0 for default behavior: it blocks until data is received, reads the data from the internal socket buffer, copies it to our buffer, and removes it from the socket buffer (to avoid duplicate reception in the future). It returns the number of bytes received, -1 if it fails, or 0 if the client closes the connection.
Sending data to Client
// res is of type std::string and contains data to send.
if (send(client_fd, res.c_str(), res.size(), 0) < 0)
{
std::cerr << "Error sending reply: " << res << "\n";
}
Enter fullscreen mode Exit fullscreen mode
We use the send() function to send data to the client. It needs the client file descriptor to identify which client we are talking to, the data to send (as a C-style string - char *), and the size of the data being sent. It returns the number of bytes sent, or -1 if it fails.
Continuously handling requests
Now it's not like a client would only make one request. They can make multiple requests. So we need to handle them continuously. We can do this by using a while loop. I will call the function handleClient(), putting together the recv and send functions inside the loop.
void handleClient(int client_fd)
{
while (true)
{
char buffer[8193]{};
int bytes = recv(client_fd, buffer, sizeof(buffer) - 1, 0);
if (bytes < 0)
{
std::cerr << "Error receiving message \n";
break;
}
if (bytes == 0)
{
std::cerr << "Client Disconnected \n";
break;
}
buffer[bytes] = '\0';
/**
Parsing request
(Ignore this for now I will explain it in the next blog post)
**/
HttpRequest req = parse_request(buffer);
auto error = isValidRequest(req);
HttpResponse res{};
if (error)
{
res = error.value();
}
else
{
res = route(req);
}
/**
Building Appropriate HTTP Response
(Ignore this for now I will explain it in the next blog post)
**/
std::string http_res = build_response(res);
std::cout << "Received:\n" << buffer << "\n";
std::cout << "Sending:\n" << http_res << "\n";
if (send(client_fd, http_res.c_str(), http_res.size(), 0) < 0)
{
std::cerr << "Error sending reply: " << http_res << "\n";
}
}
close(client_fd);
std::cout << "Client: " << client_fd << " Has Disconnected\n";
}
Enter fullscreen mode Exit fullscreen mode
I know the above code is a lot compared to the code snippet for receiving and sending data. What is actually happening here? Well, I use a while loop so the client can send requests continuously without closing the connection. Remember that recv is a blocking call, meaning it will stop until it gets data from the client or the client closes the connection. So after it receives data and stops blocking, I check if receiving failed (bytes < 0) or if the client left (bytes == 0), in which cases the while loop breaks. Otherwise, I explicitly null-terminate the buffer (buffer[bytes] = '\0') so functions parsing C-strings can read it safely. Finally, I do close(client_fd) to close the connection when the loop breaks.
After successfully receiving the data, I call the function parse_request(buffer) which parses the request and stores it in an HttpRequest struct. I will explain this function in the next blog post. Then I check if the request is valid using isValidRequest(req). If the request is invalid, I send an error response using send_error_response(). Otherwise, I route the request to the appropriate handler using route(req). Finally, I build the HTTP response using build_response(res) and send it to the client using send(client_fd, res.c_str(), res.size(), 0). And that's it.
Receive -> Parse -> Validate -> Build Response -> Send
Connecting everything
Now that we have all the required functions, let's put it all together in the main() function.
int main()
{
int server_fd = startServer();
if (server_fd < 0)
{
return 1;
}
std::cout << "Server started on port 8080" << std::endl;
while (true)
{
int client_fd = getClient(server_fd);
if (client_fd < 0)
{
continue;
}
handleClient(client_fd);
}
// The below code would never execute as the while loop runs indefinitely.
close(server_fd);
return 0;
}
Enter fullscreen mode Exit fullscreen mode
We use a while loop so our server runs indefinitely, waiting for clients. (Note: In this initial part, handleClient() processes connections sequentially on a single thread. In future parts of this series, we will make the server handle multiple client connections concurrently using multithreading)
Want to test the server?
Well, just remove the part of the code commented as "Ignore for now" and change the arguments of send() to buffer and bytes instead of http_res.c_str() and http_res.size() respectively.
Name the file server.cpp.
Now use this command to generate a binary file and to run it in one terminal:
g++ -std=c++17 -Wall server.cpp -o server
Enter fullscreen mode Exit fullscreen mode
./server
Enter fullscreen mode Exit fullscreen mode
Then type nc localhost 8080 in another terminal and then type anything like Hello! in that terminal and press enter. You should see the same text being printed back in your terminal. It will look like the terminal is "stuck" or "waiting" after you type; this is normal as the connection is still open and the server is waiting for more requests. To close the connection, press Ctrl+C.
What's Next
In the next blog post, I will be explaining Parsing HTTP Requests and Building HTTP Responses.
0 Comments
Log in to join the conversation.No comments yet. Be the first to share your thoughts.