samedi 6 novembre 2021

Curl paralell download returns empty files halfway through download

curl 'website.com/object.php?shop_id=32&page=[1-40]&PHPSESSID=phpsessid' -O --parallel --parallel-max 5 --parallel-immediate

The command is working and downloads all 40 pages of the page but halfway through the progress it starts to return empty files for the remaining half of the files where as removing the paralell options returns all the content without leaving it empty. The Verbose option is not of much help, the only difference from all the requests that is displayed for the last working request is

" * Connection cache is full, closing the oldest one.
* Closing connection 2
* Connection #4 to host website.com left intact"

After that all the remaining downloads fail and curl just stores empty files. Again, Removing --parallel --parallel-max 5 --parallel-immediate from the request solves the issue but I need the fast download speed.

What may be the reason?




Aucun commentaire:

Enregistrer un commentaire