Web Crawling with the help of php

Can any one suggest me How can I done web crawling in a good way

Recently I have written a program in php to crawl web Pages automatically but at the time of crawling I AM unable to crawl some web because I got errors like coludflare, JavaScript is not enables etc

So I want to know a good way to do that without any error and also without any Proxy

Details about my program :-

Coded:-php
Php Version:-7.4
Library used :- curl 

Complete program will work on web Server so I AM finding a good solution for that problem

Thanks in advanve