system
Given the following post, respond as if you're commenting on Reddit:
human
Been trying to solve this problem for the last couple of days but it seems impossible to solve

Hey all, last couple of days have been nothing but pure annoyance and rage for me. Decided to ask here for help maybe an experienced developer can help me out with this...

Basically, the problem is that I have a NodeJS server(NextJS to be exact) and when a user makes a request to a specific endpoint, I want to generate a bunch od PDF files(around 20-30kb in size each, in total even up to a few thousand files in some cases, but in some cases only a few), then I want to put them inside a .zip file(just for the convenience really) and serve them to the user somehow..

My initial solution was to just do all that in memory and in sync. I knew before even implementing it that this won't hold for long. Sure enough, even with 30-50 pdf files it was timing out the requests and extremely slow. But I wanted to implement it just to see if it's even possible to do what I want to do, and it was. Good start.

Next, I decided that it would be great if I made the zip in memory, but instead of sending it straight to the user, I would upload it to Azure and then send the user a link to the file. This.. didn't really work.. It was still working in a synchronous manner, so the server would just hang when you wanted to export the file and be completely unusable until it uploaded the file to Azure.

Fair enough, it's time to put this in a background task. I installed the `agenda` library thinking that this would be the final thing I needed to do.. The problem with this was, the app was being hosted on Vercel and even though the job ran in the background, the request still timed out. Even though I returned a response to the user right after clicking the export button, Vercel was giving me a "request timed out" error every time I tried this. I am really not sure why that was the case, it seems like that shouldn't happen.. 

My next idea was to have a separate server running on Azure App Service whos only task would be to zip and upload these files. But this idea soon fell in the water when I noticed that I would have to copy a lot of the code over, and some of it, like my database models, would not be easy to maintain and update, since I would have to do it in 2 places every time I make any changes...

Anyways, the last 2 days I've been trying to migrate my **entire** app to Azure App Service, and since NextJS wasn't really made to be hosted there, it's been painful. Constant crashes, errors and issues that I have no idea what to do with. And I honestly don't want to waste any more time on this since I am not sure if that is even going to work... So that's why I'm here. I would love for someone to tell me how the hell should I solve this because I'm just sick of it.. Thanks for any comments and/or suggestions... I feel like this kind of problem would not be a first for many people, and having some professional experience with software development would really help out a lot, but as a mere student, I hope that you understand that I don't really know what I'm doing lol.. Thanks again!
gpt
You should not be generating a thousand PDFs in an on demand manner while the user waits.  

A UI might trigger the request; then you want to do some form of push from the server to the UI to let them know that they are ready to be downloaded.