The Crystal Programming Language Forum

HTTP::Client stream body as IO

How can I stream a body as an IO with HTTP::Client, that is sent to a remote server?
From what I saw, it is not currently possible - an intermediary IO is copied to the socket IO.

Check out HTTP::FormData. I’d link it but I’m on my phone. :slightly_smiling_face: It’s in the stdlib, though. I just discovered it the other day.

No, it’s not related. This is for parsing a multipart/form-data response, not sending data as an IO stream.

What exactly are you looking for, if not IO.copy? That does exactly do what you asked for.

I was wondering if I can directly write to the socket IO, without any intermediary IO?
In my case, this is for streaming JSON to a remote server.
So instead of all JSON -> IO copy to -> Socket IO, doing JSON -> stream to -> Socket IO.

The reason this can’t be provided is because sending a request can either be with a fixed content-length or chunked. In the case of content-length the length must be specified before the body. Maybe you know the length and can stream directly? Maybe. But then if it’s chunked, which is what you probably want, each chunk length comes before the chunk and so it’s basically the same problem. In Go it’s also the same: when you post a reader you provide an IO to read from, then the client reads chunks from it, write the length and then the chunk. It’s a bit impossible to let you do the writing.

1 Like

I see, thank you for the explanation @asterite!
HTTP isn’t suited for this.

You can actually use a pipe in order to have a write IO. It adds a little overhead though.

Technically, I think it should be possible to provide access to the (write) IO directly. When wrapped in a buffered writer, it would be perfectly fine to use a chunked transfer encoding.

1 Like