Whats important initial page load speed or the cache load speed?

So I am trying to decrease the page load speed of a website and wanted to know if I should worry about my initial load time or just the second cache load time. I noticed while testing that my website loads faster the second time than the first. I know initial load time is important. Im just wondering if I got my page to at most 3 seconds to load first time and 1.30 second the second time is that satisfactory?