ERR_HTTP_HEADERS_SENT from onProxyReq with keep-alive #472

Activity

1. nickwesselman commented on Sep 20, 2020

   

   nickwesselman

   on Sep 20, 2020

   Author

   More actions

   This appears to be an issue with node-http-proxy, and the fact that it uses .on('socket') to fire the proxyReq event. This took a lot of digging into the behavior of HTTP requests in node. When a socket is immediately available to a request, the event fires in the ClientRequest constructor, so the request headers can be modified. But with keepAlive enabled on the agent, a socket may not be immediately available. In that case the request continues on and begins buffering its output, so by the time onSocket is invoked, the request header has been 'sent'.

   I was able to reproduce this without use of http-proxy:

   const express = require('express');
   const http = require('http');
   
   const agent = new http.Agent({
       keepAlive: true,
       maxSockets: 5,
       maxFreeSockets: 5
   });
   
   const _originalOnSocket = http.ClientRequest.prototype.onSocket;
   http.ClientRequest.prototype.onSocket = function onSocket(socket, err) {
       console.log(`Received socket, are headers sent? ${this.headersSent}`);
       const myObject = {};
       Error.captureStackTrace(myObject);
       console.log(myObject.stack);
       _originalOnSocket.apply(this, arguments);
   }
   
   const server = express();
   
   server.get('/', function (req, res, next) {
       var httpOptions = {
           hostname: "localhost",
           path: "/",
           agent
       };
       
       var proxyReq = http.request(httpOptions);
       proxyReq.on('socket', (socket) => {
           console.log(`Setting header`);
           proxyReq.setHeader('TEST', 'hello world');   
       });
   
       req.pipe(proxyReq);
   
       proxyReq.on('response', (proxyRes) => {
           res.status = proxyRes.status;
           proxyRes.pipe(res);
       });
   
   });
   
   server.listen(3000);
   

   Output on failure

   Received socket, are headers sent? true
   Error
       at ClientRequest.onSocket (C:\dev\nwe-jss-test\jss13.0.1-sc9.3.0\keepalive-middleware-test\just-node.js:14:11)
       at setRequestSocket (_http_agent.js:428:7)
       at Agent.<anonymous> (_http_agent.js:95:7)
       at Agent.emit (events.js:315:20)
       at Socket.onFree (_http_agent.js:315:11)
       at Socket.emit (events.js:315:20)
       at emitFreeNT (_http_client.js:667:10)
       at processTicksAndRejections (internal/process/task_queues.js:83:21)
   Setting header
   _http_outgoing.js:518
       throw new ERR_HTTP_HEADERS_SENT('set');
   

   So this isn't a problem with http-proxy-middleware, but does anyone perhaps have a suggestion as to an alternate way to set headers dynamically based on values from the incoming request?

   👍React with 👍3Reacted by David Adam Coffey, Todmy and Julien Viala

   🚀React with 🚀2Reacted by Victor Algaze and Mohseen Mukaddam
2. nickwesselman commented on Sep 25, 2020

   

   nickwesselman

   on Sep 25, 2020

   Author

   More actions

   Workaround we've found is to instead modify the request headers in separate middleware, before applying the proxy.

   server.use((req, res, next) => {
     // change req here
     next();
   });
   
3. bobalong79 commented on Feb 28, 2021

   

   bobalong79

   on Feb 28, 2021

   More actions

   I've been having the same issue and I think it happens when any agent is used, not something specific to keepalive (I'm using global-agent's HttpsProxyAgent). My workaround is to use the custom filter mechanism to alter the headers prior to the proxy request being set up. It looks like this filter mechanism was designed to allow you to implement custom matching logic, but as it provides access to the req: IncomingRequest object before the proxy request is created it is a convenient way to manipulate the headers before they even reach the http-proxy code.

   const filter = (_pathname: string, req: http.IncomingMessage): boolean => {
     req.headers['TEST'] = 'hello world';
   
     return true;
   };
   
   server.use('*', createProxyMiddleware(filter, {
     agent,
     target: apiHost,
     changeOrigin: true
   }));
   

   I think this is basically the same as what you've done but it keeps all the logic in the same middleware.

   I guess one drawback of this approach (and your workaround) is that because the headers are changed on the original incoming request rather than on the proxied request, other downstream routers that rely on "untampered" headers might be compromised. In that case then you'd need to keep a copy of the original set and copy them back after onProxyRes completes. I don't need to do that for my application though.

   👍React with 👍5Reacted by JounQin, Matteo Barone, Teric, Yudu and Mohseen Mukaddam

   ❤️React with ❤️2Reacted by Constantin and Mohseen Mukaddam
4. JounQin commented on Sep 14, 2021

   

   JounQin

   on Sep 14, 2021

   More actions

   I also meet this issue when using fixRequestBody from this middleware.

   👍React with 👍3Reacted by Mark, Teric and Todmy
5. 3rj commented on Sep 21, 2021

   

   3rj

   on Sep 21, 2021

   More actions

   I also meet this issue when using fixRequestBody from this middleware.

   Did you find any workaround to this issue. (using fixRequestBody and agent together)
6. JounQin commented on Sep 21, 2021

   

   JounQin

   on Sep 21, 2021

   More actions

   I also meet this issue when using fixRequestBody from this middleware.

   Did you find any workaround to this issue. (using fixRequestBody and agent together)

   Nope, I have to move proxy middlewares to be used firstly.
7. markcellus commented on Mar 29, 2022

   

   markcellus

   on Mar 29, 2022

   More actions

   Also ran into this issue with fixRequestBody 😞
8. chestnutchen commented on Jul 20, 2023

   

   chestnutchen

   on Jul 20, 2023

   More actions

   Also ran into this issue with fixRequestBody
9. josslzr commented on Aug 22, 2023

   

   josslzr

   on Aug 22, 2023

   Last edited by josslzr

   More actions

   For those who still facing this issue I've found a workaround. Basically you need to set the headers and buffer properties while creating the proxy middleware:

    getProxyConfig(headers: any, body: string): RequestHandler {
       return createProxyMiddleware({
         ...//other options
         headers,
         buffer: this.getBodyStream(body)
       })
     }
     
     
      private getBodyStream(body: string): Readable {
       const stream = new Readable();
       stream.push(body);
       stream.push(null);
       return stream;
     }
   

   Hope this helps! :)
10. kevjames3 commented on Jan 30, 2025

    

    kevjames3

    on Jan 30, 2025

    More actions

    Thanks to @josslzr for providing the initial inspiration on this as their approach was actually the correct on to take. That being said, I was confused by the implementation for a bit, so I want to put my own take here as my solution required me getting access to the body and fixRequestBody was giving me grief.

    Assuming that you are setting this up in express somewhere to process the body...

      app.use(express.json());

    And then for the middleware

    export const proxyMiddlewareWrapper = (
      payload: proxyPayload,
    ): RequestHandler => {
      return (req: Request, res: Response, next: NextFunction) => {
        const contentType = req.header('Content-Type');
        const header = {} as Record<string, string>;
    
        if (contentType) {
          header['Content-Type'] = contentType;
          header['Content-Length'] = String(
            Buffer.byteLength(JSON.stringify(req.body)),
          );
        }
    
        const stream = new Readable();
        stream.push(JSON.stringify(req.body));
        stream.push(null);
    
        createProxyMiddleware({
          // ...
          headers: header,
          buffer: stream,
          // ...
          on: {
            proxyReq: async (
              proxyReq: http.ClientRequest,
              req: Request,
              res: Response,
            ) => {
              // This should be populated as normal
                const body = req.body as unknown;
            },
          },
        })(req, res, next);
      };
    };

    Explanation:

    1. We are assuming that we are processing the body before hitting this middleware using Express's body parser (JSON version). This ensures that req.body will be populated
    2. Before setting up the proxy middleware, we will process the body into a buffer and then also add to the headers that will be sent to the downstream
    3. We then create the middleware with the params of the buffer and headers and then immediately pass in the called surrounding middleware params making sure we only call the middleware when all params are ready.

    This makes sure we have no more race condition and that the proxy middleware pulls from the buffer that we specify while having access to the body
11. bc2009 commented on Mar 7, 2025

    

    bc2009

    on Mar 7, 2025

    Last edited by bc2009

    More actions

    Thanks to @josslzr for providing the initial inspiration on this as their approach was actually the correct on to take. That being said, I was confused by the implementation for a bit, so I want to put my own take here as my solution required me getting access to the body and fixRequestBody was giving me grief.

    Assuming that you are setting this up in express somewhere to process the body...

    app.use(express.json());
    And then for the middleware

    export const proxyMiddlewareWrapper = (
    payload: proxyPayload,
    ): RequestHandler => {
    return (req: Request, res: Response, next: NextFunction) => {
    const contentType = req.header('Content-Type');
    const header = {} as Record<string, string>;

    if (contentType) {
      header['Content-Type'] = contentType;
      header['Content-Length'] = String(
        Buffer.byteLength(JSON.stringify(req.body)),
      );
    }
    
    const stream = new Readable();
    stream.push(JSON.stringify(req.body));
    stream.push(null);
    
    createProxyMiddleware({
      // ...
      headers: header,
      buffer: stream,
      // ...
      on: {
        proxyReq: async (
          proxyReq: http.ClientRequest,
          req: Request,
          res: Response,
        ) => {
          // This should be populated as normal
            const body = req.body as unknown;
        },
      },
    })(req, res, next);
    

    };
    };

    Explanation:

    1. We are assuming that we are processing the body before hitting this middleware using Express's body parser (JSON version). This ensures that req.body will be populated
    2. Before setting up the proxy middleware, we will process the body into a buffer and then also add to the headers that will be sent to the downstream
    3. We then create the middleware with the params of the buffer and headers and then immediately pass in the called surrounding middleware params making sure we only call the middleware when all params are ready.

    This makes sure we have no more race condition and that the proxy middleware pulls from the buffer that we specify while having access to the body

    @kevjames3 , can you post a complete working example? In your proxyReq you have a oneliner for body, what do u do after you have the body, do you need to use proxyReq.write() to write it out?

    I have the same same error with this discussion. But when I try your code, for me, the proxy looks to get stuck waiting until the sever reset (probably due to timeout cutoff). If I add proxyReq.end(), I will get an error:

    Error [ERR_STREAM_WRITE_AFTER_END]: write after end
        at RedirectableRequest.write. (... node_modules/follow-redirects/index.js:120:11)
        at Readable.ondata (node:internal/streams/readable:754:22)
        at Readable.emit (node:events:513:28)
        at Readable.read (node:internal/streams/readable:527:10)
        at flow (node:internal/streams/readable:1011:34)
        at resume_ (node:internal/streams/readable:992:3)
        at processTicksAndRejections (node:internal/process/task_queues:83:21) {
      code: 'ERR_STREAM_WRITE_AFTER_END'
    }
    

    My use case is that my target server is an HTTP2 stream service which probably is a proxy either to some internal service or cloud service at like Azure (which is decided by a field in my request).

    My case is when proxying to internal service, this proxy middleware works ok. But when proxying to use external cloud like Azure, the setHeader will fail as described at the top of this issue.

    Not sure if the HTTP/2 stream protocol will have similiar effect like keep-alive discussed here (from HTTP/2 rfc looks like connection is persistent). I am thinking to try this http-party/node-http-proxy#1478 (comment) directly to spin up my own proxy server to do customization.
12. chimurai commented on Jun 18, 2026

    

    chimurai

    on Jun 18, 2026

    Last edited by chimurai

    Owner

    More actions

    Fixed in v4.1, which uses httpxy (unjs/httpxy#111)

    Issue was reproducible in v1, v2 and v3 (http-proxy bug)

    Feel free to re-open a new issue if you are still experiencing issues in the latest version

    🎉React with 🎉1Reacted by Nick Wesselman
13. chimurai

    closed this as completedon Jun 18, 2026
14. chimurai

    linked a pull request that will close this issuechore(package.json): update to httpxy@0.5.3 #1234on Jun 18, 2026
15. chimurai

    added a commit that references this issue on Jun 18, 2026

    test(e2e): add HttpAgent keepAlive test (#472)

    66948bc
16. chimurai

    linked a pull request that will close this issuetest(e2e): add HttpAgent keepAlive test (#472) #1267on Jun 18, 2026
17. chimurai

    added a commit that references this issue on Jun 18, 2026

    test(e2e): add HttpAgent keepAlive test (#472) (#1267)

    22281bf
18. karelhala

    added a commit that references this issue on Jul 9, 2026

    fix(proxy): upgrade http-proxy-middleware and add headersSent guards (#…

    ...

    7bdb956
19. pi0x

    added a commit that references this issue on Sep 15, 2026

    fix(web-incoming): defer body piping until socket is assigned

    ...

    10a916b