Pointing ports on the same domain to two different IP addresses

I have two servers on two completely different IP addresses, and I'm trying to get one domain to point to different ports on each.

I'm trying to get {domain}.com:80 and {domain}.com:64738 to point to the first server, which is hosting a web site and a Mumble server on those ports respectively.

Then I want every other port on the domain to just point to the other server, where I'm hosting some other stuff.

How would I go about this? From research I figure it's something to do with SRV records, but I'm not sure how. This is all I have in the way of SRV records from the domain name control panel: image Thanks!