Do I need an HTML file for every single page on my website?

Say I have a product website, like Amazon (this is not the case, but it will help me explain my point), and I have a URL for every single product (such as with Amazon)... Do I need to copy-paste and modify an HTML file for every single individual product page, or is there a way to use a "model" on which I can base all my other pages without recopying the whole code and modifying a few things in each? I've just started learning HTML and web development, so bare with me if I'm asking a stupid question. It just seems odd to me that a million-page website should host a million+ individual, nearly identical, HTML files.

Thank you very much in advance.

P.S. I'm using Amazon's brand name as an example here, and am not affiliated with anything related to it. Thank you for understanding.