Why is my TTFB over 10 seconds when I leave the "www." out of my URL?

I developed a wordpress site for a client of mine but he has been disappointed about the time it takes to load. Using Chrome Dev tools I found that the TTFB(Time To First Byte) is about 9 to 15 seconds. Which is huge considering most of my sites are always under a second. But I also noticed that if I include "www." in the URL the TTFB is right where it should be.

Since most people don't normally type "www." when visiting a URL this is problematic for any users vising the site. Any suggestions on what is causing this would be greatly appreciated.

Site info: URL: http://ift.tt/2lMNP9P URL without "www.": givermarketing.com Hosting Company: A2hosting Domain Company: Godaddy CMS: Wordpress Server type: Linux (.htaccess) PHP Version: 5.5.38 PHP info Link: /phpinfo.php