CGI HTTP Web Server
When a user (client) wants to access, modify, or perform certain actions on a web application, there are many steps that get involved from the browser to the server/backend (and vice versa). To understand more about the actions involved, August Eggers and I developed and demonstrated the use of the Common Gateway Interface (CGI) to communicate back and forth between the client and server based on RFC/1945. The purpose of this project is for admins to see how to handle POST and GET requests on an executable web server; programming the server helps us understand how to send HTTP headers and how to communicate with a web page, as well as browsing for files and requesting them. For the users, the purpose of this project is to learn how to navigate web pages and what certain errors mean when received. For example, if trying to load a file from the server that doesn’t exist, there is going to be a HTTP 404 Not Found error.
As seen by the diagram above, if a user wants to request a page containing images to display on their web application screen, the browser sends multiple HTTP requests to the web server, where the server passes the data to the CGI program via the CGI gateway. The CGI program then talks to the database to collect any information that is needed to be retrieved, then it passes the information back to the server, and finally the server sends HTTP responses back to the web browser. CGI scripts allow a website to become more interactive through things such as forms that will pass data to an executable (any kind of executable) and that executable will send back HTML to the client socket from which it was passed.
We implemented our server to deal with HTTP GET and POST methods while documenting the success and error logs and sending back full HTTP responses, which include the HTTP code status, Content-Type, Content-Length, Expires, and much more other information, just like a real server does. When a client requests our web page that is made up of HTML containing other graphics, such as images, GIFs, videos, etc., each page request may turn out to be many iterative GET calls to the server where the first request is for the HTML and the subsequent calls are for the images. For this CGI HTTP Web Server project, we rely heavily on C socket programming and the abstraction that we have learned in our Network Programming class to make the server efficient with regards to speed and resource allocation for the dynamic web interactions.
Problem: Our web page is made up of different images.
Result: Our web page makes many subsequent GET calls to the web server/CGI program, then the web page combines all of the images to render/load up this entire page:
Contribution
- Implement the CGI and web server, GET and POST methods
- Properly send correct HTTP status codes and headers along with possible graphics (images/GIFs) back to the client from the GET requests
- Properly accept and compute data based on the users’ POST requests
- Log messages to the server to determine if each request has been a success or a failure
- Make the server accept and respond to requests concurrently
- Document the overall goals and steps to successfully run from the GitHub repository
Lessons Learned
fork()lets us create processes; this is very helpful when we have new requests coming to the server, where each request can run independently from the other- When a client sends a POST or GET request to the server, the server will call the CGI executable to handle the request. The main use for this is to allow the executable to handle the work while the server just passes it off and can then handle new requests. This is a good use of resource allocation and helps support more users concurrently.
- HTTP statuses help us identify what could be the cause of the problem if the page does not load properly
- A page with many images has many iterative GET calls to the server, where the first request is for the HTML content and the subsequent calls are for the images
- Always log your server for future reference in case we need to debug or track certain success or failure actions
- Web browsers sometimes do not display properly if the HTTP response headers are properly configured