This is NOT ACTUALLY an input range! It is too different. Historical mistake kinda.
The main interface with the web request
A helper test class for request handler unittests.
thrown when a connection is closed remotely while we waiting on data from it
To use this thing:
WEBSOCKET SUPPORT:
If websocketRequested, you can call this to accept it and upgrade the connection. It returns the new WebSocket object you use for future communication on this connection; the cgi object should no longer be used.
This is the function GenericMain calls. View its source for some simple boilerplate you can copy/paste and modify, or you can call it yourself from your main.
breaks down a url encoded string
breaks down a url encoded string, but only returns the last value of any array
Default host for listening. 127.0.0.1 for scgi, null (aka all interfaces) for all others. If you want the server directly accessible from other computers on the network, normally use null. If not, 127.0.0.1 is a bit better. Settable with default handlers with --listening-host command line argument.
Returns the default listening port for the current cgi configuration. 8085 for embedded httpd, 4000 for scgi, irrelevant for others.
use this for testing or other isolated things when you want it to be no-ops
url encodes the whole string
url encodes a whole string
Makes a data:// uri that can be used as links in most newer browsers (IE8+).
Encodes all but the explicitly unreserved characters per rfc 3986 Alphanumeric and -_.~ are the only ones left unencoded name is borrowed from php
Tries to simulate a request from the command line. Returns true if it does, false if it didn't find the args.
Returns true if the request headers are asking for a websocket upgrade.
You can customize your server by subclassing the appropriate server. Then, register your subclass at compile time with the registerEventIoServer template, or implement your own main function and call it yourself.
Base for storing sessions in an array. Exists primarily for internal purposes and you should generally not use this.
Allows for a generic DispatcherMain with custom arguments. Note you can use defaultMaxContentLength as the second argument if you like.
If you want to use a subclass of Cgi with generic main, use this mixin.
Boilerplate mixin for a main function that uses the dispatcher function.
If you are doing a custom cgi class, mixing this in can take care of the required constructors for you
Use this instead of writing your own main
The stack size when a fiber is created. You can set this from your main or from a shared static constructor to optimize your memory use if you know you don't need this much space. Be careful though, some functions use more stack space than you realize and a recursive function (including ones like in dom.d) can easily grow fast!
A server control and configuration struct, as a potential alternative to calling GenericMain or cgiMainImpl. See the source of cgiMainImpl for a complete, up-to-date, example of how it is used internally.
Represents a url that can be broken down or built up through properties
cgi.d tries to be flexible to meet your needs. It is possible to configure it both at runtime (by writing your own main function and constructing a RequestServer object) or at compile time using the version switch to the compiler or a dub subConfiguration.
If you are using dub, use:
subConfiguration "arsd-official:cgi" "VALUE_HERE"
or to dub.json:
"subConfigurations": {"arsd-official:cgi": "VALUE_HERE"}
to change versions. The possible options for VALUE_HERE are:
With dmd, use:
Interfaces | (mutually exclusive) |
---|---|
-version=plain_cgi | The default building the module alone without dub - a traditional, plain CGI executable will be generated. |
-version=embedded_httpd | A HTTP server will be embedded in the generated executable. This is default when building with dub. |
-version=fastcgi | A FastCGI executable will be generated. |
-version=scgi | A SCGI (SimpleCGI) executable will be generated. |
-version=embedded_httpd_hybrid | A HTTP server that uses a combination of processes, threads, and fibers to better handle large numbers of idle connections. Recommended if you are going to serve websockets in a non-local application. |
-version=embedded_httpd_threads | The embedded HTTP server will use a single process with a thread pool. (use instead of plain embedded_httpd if you want this specific implementation) |
-version=embedded_httpd_processes | The embedded HTTP server will use a prefork style process pool. (use instead of plain embedded_httpd if you want this specific implementation) |
-version=embedded_httpd_processes_accept_after_fork | It will call accept() in each child process, after forking. This is currently the only option, though I am experimenting with other ideas. You probably should NOT specify this right now. |
-version=stdio_http | The embedded HTTP server will be spoken over stdin and stdout. |
Tweaks | (can be used together with others) |
-version=cgi_with_websocket | The CGI class has websocket server support. (This is on by default now.) |
-version=with_openssl | not currently used |
-version=cgi_embedded_sessions | The session server will be embedded in the cgi.d server process |
-version=cgi_session_server_process | The session will be provided in a separate process, provided by cgi.d. |
For example,
For CGI, dmd yourfile.d cgi.d then put the executable in your cgi-bin directory.
For FastCGI: dmd yourfile.d cgi.d -version=fastcgi and run it. spawn-fcgi helps on nginx. You can put the file in the directory for Apache. On IIS, run it with a port on the command line (this causes it to call FCGX_OpenSocket, which can work on nginx too).
For SCGI: dmd yourfile.d cgi.d -version=scgi and run the executable, providing a port number on the command line.
For an embedded HTTP server, run dmd yourfile.d cgi.d -version=embedded_httpd and run the generated program. It listens on port 8085 by default. You can change this on the command line with the --port option when running your program.
If using GenericMain or DispatcherMain, an application using arsd.cgi will offer a command line interface out of the box.
See RequestServer.listenSpec for more information.
If you are using one of the GenericMain or DispatcherMain mixins, or main with your own call to RequestServer.trySimulatedRequest, you can simulate requests from your command-ine shell. Call the program like this:
./yourprogram GET / name=adr
And it will print the result to stdout instead of running a server, regardless of build more..
On Apache, you may do SetHandler cgi-script in your .htaccess file to set a particular file to be run through the cgi program. Note that all "subdirectories" of it also run the program; if you configure /foo to be a cgi script, then going to /foo/bar will call your cgi handler function with cgi.pathInfo == "/bar".
cgi.d offers both lower-level handler apis as well as higher-level auto-dispatcher apis. For a lower-level handler function, you'll probably want to review the following functions:
Input: Cgi.get, Cgi.post, Cgi.request, Cgi.files, Cgi.cookies, Cgi.pathInfo, Cgi.requestMethod, and HTTP headers (Cgi.headers, Cgi.userAgent, Cgi.referrer, Cgi.accept, Cgi.authorization, Cgi.lastEventId)
Output: Cgi.write, Cgi.header, Cgi.setResponseStatus, Cgi.setResponseContentType, Cgi.gzipResponse
Cookies: Cgi.setCookie, Cgi.clearCookie, Cgi.cookie, Cgi.cookies
Caching: Cgi.setResponseExpires, Cgi.updateResponseExpires, Cgi.setCache
Redirections: Cgi.setResponseLocation
Other Information: Cgi.remoteAddress, Cgi.https, Cgi.port, Cgi.scriptName, Cgi.requestUri, Cgi.getCurrentCompleteUri, Cgi.onRequestBodyDataReceived
Websockets: Websocket, websocketRequested, acceptWebsocket. For websockets, use the embedded_httpd_hybrid build mode for best results, because it is optimized for handling large numbers of idle connections compared to the other build modes.
Overriding behavior for special cases streaming input data: see the virtual functions Cgi.handleIncomingDataChunk, Cgi.prepareForIncomingDataChunks, Cgi.cleanUpPostDataState
A basic program using the lower-level api might look like:
1 import arsd.cgi; 2 3 // you write a request handler which always takes a Cgi object 4 void handler(Cgi cgi) { 5 /+ 6 when the user goes to your site, suppose you are being hosted at http://example.com/yourapp 7 8 If the user goes to http://example.com/yourapp/test?name=value 9 then the url will be parsed out into the following pieces: 10 11 cgi.pathInfo == "/test". This is everything after yourapp's name. (If you are doing an embedded http server, your app's name is blank, so pathInfo will be the whole path of the url.) 12 13 cgi.scriptName == "yourapp". With an embedded http server, this will be blank. 14 15 cgi.host == "example.com" 16 17 cgi.https == false 18 19 cgi.queryString == "name=value" (there's also cgi.search, which will be "?name=value", including the ?) 20 21 The query string is further parsed into the `get` and `getArray` members, so: 22 23 cgi.get == ["name": "value"], meaning you can do `cgi.get["name"] == "value"` 24 25 And 26 27 cgi.getArray == ["name": ["value"]]. 28 29 Why is there both `get` and `getArray`? The standard allows names to be repeated. This can be very useful, 30 it is how http forms naturally pass multiple items like a set of checkboxes. So `getArray` is the complete data 31 if you need it. But since so often you only care about one value, the `get` member provides more convenient access. 32 33 We can use these members to process the request and build link urls. Other info from the request are in other members, we'll look at them later. 34 +/ 35 switch(cgi.pathInfo) { 36 // the home page will be a small html form that can set a cookie. 37 case "/": 38 cgi.write(`<!DOCTYPE html> 39 <html> 40 <body> 41 <form method="POST" action="set-cookie"> 42 <label>Your name: <input type="text" name="name" /></label> 43 <input type="submit" value="Submit" /> 44 </form> 45 </body> 46 </html> 47 `, true); // the , true tells it that this is the one, complete response i want to send, allowing some optimizations. 48 break; 49 // POSTing to this will set a cookie with our submitted name 50 case "/set-cookie": 51 // HTTP has a number of request methods (also called "verbs") to tell 52 // what you should do with the given resource. 53 // The most common are GET and POST, the ones used in html forms. 54 // You can check which one was used with the `cgi.requestMethod` property. 55 if(cgi.requestMethod == Cgi.RequestMethod.POST) { 56 57 // headers like redirections need to be set before we call `write` 58 cgi.setResponseLocation("read-cookie"); 59 60 // just like how url params go into cgi.get/getArray, form data submitted in a POST 61 // body go to cgi.post/postArray. Please note that a POST request can also have get 62 // params in addition to post params. 63 // 64 // There's also a convenience function `cgi.request("name")` which checks post first, 65 // then get if it isn't found there, and then returns a default value if it is in neither. 66 if("name" in cgi.post) { 67 // we can set cookies with a method too 68 // again, cookies need to be set before calling `cgi.write`, since they 69 // are a kind of header. 70 cgi.setCookie("name" , cgi.post["name"]); 71 } 72 73 // the user will probably never see this, since the response location 74 // is an automatic redirect, but it is still best to say something anyway 75 cgi.write("Redirecting you to see the cookie...", true); 76 } else { 77 // you can write out response codes and headers 78 // as well as response bodies 79 // 80 // But always check the cgi docs before using the generic 81 // `header` method - if there is a specific method for your 82 // header, use it before resorting to the generic one to avoid 83 // a header value from being sent twice. 84 cgi.setResponseLocation("405 Method Not Allowed"); 85 // there is no special accept member, so you can use the generic header function 86 cgi.header("Accept: POST"); 87 // but content type does have a method, so prefer to use it: 88 cgi.setResponseContentType("text/plain"); 89 90 // all the headers are buffered, and will be sent upon the first body 91 // write. you can actually modify some of them before sending if need be. 92 cgi.write("You must use the POST http verb on this resource.", true); 93 } 94 break; 95 // and GETting this will read the cookie back out 96 case "/read-cookie": 97 // I did NOT pass `,true` here because this is writing a partial response. 98 // It is possible to stream data to the user in chunks by writing partial 99 // responses the calling `cgi.flush();` to send the partial response immediately. 100 // normally, you'd only send partial chunks if you have to - it is better to build 101 // a response as a whole and send it as a whole whenever possible - but here I want 102 // to demo that you can. 103 cgi.write("Hello, "); 104 if("name" in cgi.cookies) { 105 import arsd.dom; // dom.d provides a lot of helpers for html 106 // since the cookie is set, we need to write it out properly to 107 // avoid cross-site scripting attacks. 108 // 109 // Getting this stuff right automatically is a benefit of using the higher 110 // level apis, but this demo is to show the fundamental building blocks, so 111 // we're responsible to take care of it. 112 cgi.write(htmlEntitiesEncode(cgi.cookies["name"])); 113 } else { 114 cgi.write("friend"); 115 } 116 117 // note that I never called cgi.setResponseContentType, since the default is text/html. 118 // it doesn't hurt to do it explicitly though, just remember to do it before any cgi.write 119 // calls. 120 break; 121 default: 122 // no path matched 123 cgi.setResponseStatus("404 Not Found"); 124 cgi.write("Resource not found.", true); 125 } 126 } 127 128 // and this adds the boilerplate to set up a server according to the 129 // compile version configuration and call your handler as requests come in 130 mixin GenericMain!handler; // the `handler` here is the name of your function
Even if you plan to always use the higher-level apis, I still recommend you at least familiarize yourself with the lower level functions, since they provide the lightest weight, most flexible options to get down to business if you ever need them.
In the lower-level api, the Cgi object represents your HTTP transaction. It has functions to describe the request and for you to send your response. It leaves the details of how you o it up to you. The general guideline though is to avoid depending any variables outside your handler function, since there's no guarantee they will survive to another handler. You can use global vars as a lazy initialized cache, but you should always be ready in case it is empty. (One exception: if you use -version=embedded_httpd_threads -version=cgi_no_fork, then you can rely on it more, but you should still really write things assuming your function won't have anything survive beyond its return for max scalability and compatibility.)
A basic program using the higher-level apis might look like:
/+
import arsd.cgi;
struct LoginData {
string currentUser;
}
class AppClass : WebObject {
string foo() {}
}
mixin DispatcherMain!(
"/assets/.serveStaticFileDirectory("assets/", true), // serve the files in the assets subdirectory
"/".serveApi!AppClass,
"/thing/".serveRestObject,
);
+/
(Please note: I wrote this section in 2008. A lot of PHP hosts still ran 4.x back then, so it was common to avoid using classes - introduced in php 5 - to maintain compatibility! If you're coming from php more recently, this may not be relevant anymore, but still might help you.)
If you are coming from old-style PHP, here's a quick guide to help you get started:
<?php $foo = $_POST["foo"]; $bar = $_GET["bar"]; $baz = $_COOKIE["baz"]; $user_ip = $_SERVER["REMOTE_ADDR"]; $host = $_SERVER["HTTP_HOST"]; $path = $_SERVER["PATH_INFO"]; setcookie("baz", "some value"); echo "hello!"; ?> | import arsd.cgi; void app(Cgi cgi) { string foo = cgi.post["foo"]; string bar = cgi.get["bar"]; string baz = cgi.cookies["baz"]; string user_ip = cgi.remoteAddress; string host = cgi.host; string path = cgi.pathInfo; cgi.setCookie("baz", "some value"); cgi.write("hello!"); } mixin GenericMain!app |
In PHP, you can give a form element a name like "something[]", and then $_POST["something"] gives an array. In D, you can use whatever name you want, and access an array of values with the cgi.getArray["name"] and cgi.postArray["name"] members.
PHP has a lot of stuff in its standard library. cgi.d doesn't include most of these, but the rest of my arsd repository has much of it. For example, to access a MySQL database, download database.d and mysql.d from my github repo, and try this code (assuming, of course, your database is set up):
import arsd.cgi; import arsd.mysql; void app(Cgi cgi) { auto database = new MySql("localhost", "username", "password", "database_name"); foreach(row; mysql.query("SELECT count(id) FROM people")) cgi.write(row[0] ~ " people in database"); } mixin GenericMain!app;
Similar modules are available for PostgreSQL, Microsoft SQL Server, and SQLite databases, implementing the same basic interface.
This micro-example uses the dispatcher api to act as a simple http file server, serving files found in the current directory and its children.
import arsd.cgi; mixin DispatcherMain!( "/".serveStaticFileDirectory(null, true) );
Same as the previous example, but written out long-form without the use of DispatcherMain nor GenericMain.
import arsd.cgi; void requestHandler(Cgi cgi) { cgi.dispatcher!( "/".serveStaticFileDirectory(null, true) ); } // mixin GenericMain!requestHandler would add this function: void main(string[] args) { // this is all the content of [cgiMainImpl] which you can also call // cgi.d embeds a few add on functions like real time event forwarders // and session servers it can run in other processes. this spawns them, if needed. if(tryAddonServers(args)) return; // cgi.d allows you to easily simulate http requests from the command line, // without actually starting a server. this function will do that. if(trySimulatedRequest!(requestHandler, Cgi)(args)) return; RequestServer server; // you can change the default port here if you like // server.listeningPort = 9000; // then call this to let the command line args override your default server.configureFromCommandLine(args); // here is where you could print out the listeningPort to the user if you wanted // and serve the request(s) according to the compile configuration server.serve!(requestHandler)(); // or you could explicitly choose a serve mode like this: // server.serveEmbeddedHttp!requestHandler(); }
cgi.d has built-in testing helpers too. These will provide mock requests and mock sessions that otherwise run through the rest of the internal mechanisms to call your functions without actually spinning up a server.
import arsd.cgi; void requestHandler(Cgi cgi) { } // D doesn't let me embed a unittest inside an example unittest // so this is a function, but you can do it however in your real program /* unittest */ void runTests() { auto tester = new CgiTester(&requestHandler); auto response = tester.GET("/"); assert(response.code == 200); }
You may also want to see arsd.dom, arsd.webtemplate, and maybe some functions from my old arsd.html for more code for making web applications. dom and webtemplate are used by the higher-level api here in cgi.d.
For working with json, try arsd.jsvar.
arsd.database, arsd.mysql, arsd.postgres, arsd.mssql, and arsd.sqlite can help in accessing databases.
If you are looking to access a web application via HTTP, try arsd.http2.
cgi.d copyright 2008-2023, Adam D. Ruppe. Provided under the Boost Software License.
Yes, this file is old, and yes, it is still actively maintained and used.
An import of arsd.core was added on March 21, 2023 (dub v11.0). Prior to this, the module's default configuration was completely stand-alone. You must now include the core.d file in your builds with cgi.d.
This change is primarily to integrate the event loops across the library, allowing you to more easily use cgi.d along with my other libraries like simpledisplay and http2.d. Previously, you'd have to run separate helper threads. Now, they can all automatically work together.
Provides a uniform server-side API for CGI, FastCGI, SCGI, and HTTP web applications. Offers both lower- and higher- level api options among other common (optional) things like websocket and event source serving support, session management, and job scheduling.
Or:
Test on console (works in any interface mode):
If using http version (default on dub builds, or on custom builds when passing -version=embedded_httpd to dmd):
Please note: the default port for http is 8085 and for scgi is 4000. I recommend you set your own by the command line argument in a startup script instead of relying on any hard coded defaults. It is possible though to code your own with RequestServer, however.