How to preview AVI/MKV video file on my web

I am creating my home web server movie database. On the site, you can click on certain movies to play them. I know that <video> tag can play MP4/OGG/WebM but most of my movies are .avi or .mkv which <video> tag does not support.

My question is how to implement these avi/mkv files on my web?

Can I convert them somehow in my HTML/JavaScript code into MP4?

Thank you.