Video conversion slow on large videos ffmpeg adult website

So I'm building an adult website and people will upload videos. The backend is built with node.js. Some videos which could potentially be uploaded may be many hundreds of mb in size. Problem is video conversion with ffmpeg is relatively slow. I am considering other options such as client side video conversion ( probably slower but far less load on my servers). I don't know a great deal about how these very big video websites handle processing the large number of videos. Anyone able to advise me on how i should best go about it? They all get converted to a range of sized mp4's. A range of gif and png thumbnails are also generated. My system is working and for a video say 10mb in size its fine but its been going over an hour now converting a 236mb video to five sizes ( different widths so i can save data on mobile devices). I have a fully working system which handles different resolutions all that converts them but the conversion is a tad too slow for me and i dont want to invest in very expensive hardware if i can help it cos this will be a startup. I was probably going to use a cdn for hosting the actual content but have designed it so i can host on my own servers within the network too. At present my system is designed to have a number of servers all with an interserver communication channel and also the ability to automatically use SFTP to share files with each other. The video gets uploaded to one of a number of designated multimedia servers and converted. Then it gets distributed to the other multimedia servers so they can all provide it. That all works great. What I mean is i can pretty much implement any solution anyone has to this problem. But I dont know what the best solution is.