THttpServer serve big binary files (bigger than RAM) on the fly (or CGI?)

[jb023]

Hi,

I’m using:

THttpServer *server = new CustomHttpServer("http:80"); 
server->AddLocation("foo/", "bar");

with a custom MissedRequest handler void CustomHttpServer::MissedRequest(THttpCallArg *arg) { to handle custom requests. It works.

Now I need the server to serve binary data:

/download/big_on_the_fly_response

which is possibly bigger than RAM. Also I want to generate this file on the fly.

1. can we serve request answer of 10 GB (bigger than RAM) in MissedRequest, by writing the request response in chunks of 1MB? If so, how? Pseudo code:

   void CustomHttpServer::MissedRequest(THttpCallArg *arg) {
        if (!strcmp(arg->GetPathName(), "...") && (!strcmp(arg->GetFileName(), "...")) {
            for (int i = 0; i < 10*1000; i++) {
                data = get_next_1_MB_chunk()
                send_chunked_response(data);
            }
            send_end_of_request_response();
       }
       ...
   
   

2. Or should I use CGI? (NB: I don’t use Apache or any other webserver, I only use THttpServer)

   server->CreateEngine("fastcgi:9000?top=fastcgiserver");
   

   Then, how to add a custom handler? Pseudo-code:

   server->addCGIHandler("/download/big_file", callback);
   

   with callback doing the work:

     for (int i = 0; i < 10*1000; i++) {
         data = get_next_1_MB_chunk()
         send_chunked_response(data);
     }
     send_end_of_request_response();
   

Linked: I have already read Github HttpServer.md#using-fastcgi-interface and THttpServer with FastCGI.

Thanks in advance! (Maybe @linev you know about this? Thanks again)

Have a good day.

[siliataider]

Thanks @jb023 for your report!
@linev maybe you can help with this?

[jb023]

Does anyone have an idea?
Maybe @linev would you know how to do this?
Thanks in advance!

[linev]

Hi,

Sorry for some delay - I was in vacations.

Most easy way to resolve your problem - generate file with all necessary information there and then let civetweb process partial HTTP requests to such file. This let you use any utility like wget or curl to download that file from client side.

THttpCallArg does not provide all required functionality to implement such partial requests processing in CustomHttpServer::MissedRequest handler.

FastCGI will not help here - it is just API to redirect http requests from “normal” http server to ROOT application.

Regards,
Sergey