jeudi 3 août 2017

How to check is webpage content is modified or not?

My actual purporse for crawling webpage

Some web pages get last modified date and some webpage not get last modified date

Not Getting Modified Date

    Accept-Ranges:bytes
    Age:0
    Content-Encoding:gzip
    Content-Length:12458
    Content-Type:text/html; charset=UTF-8
    Date:Thu, 03 Aug 2017 09:00:51 GMT
    Vary:Accept-Encoding, User-Agent
    X-Backend:all_requests
    X-Cache:uncached
    X-Cache-Hit:MISS
    X-Cacheable:YES:Forced
    X-Port:port_10128

Getting Modified Date


Accept-Ranges:bytes
Age:545944
Cache-Control:max-age=31622400
Connection:keep-alive
Content-Encoding:gzip
Content-Length:67742
Content-Type:text/css
Date:Fri, 28 Jul 2017 01:24:13 GMT
Etag:W/"597a0925-66a34"
Expires:Sun, 29 Jul 2018 01:24:13 GMT
Last-Modified:Thu, 27 Jul 2017 15:39:17 GMT




Aucun commentaire:

Enregistrer un commentaire