We are in the process of migrating this forum. A new space will be available soon. We are sorry for the inconvenience.

how to have multiple VMs accessible from the internet linux iptables


jonlewi5
04-10-2013, 14:01
I've been dealing with a similar problem myself actually and like this setup.
I've gone a different way around it.

Im using proxmox on the host. Have a failover sat on a pfsense vm and a collection of vm's behind it.
To get to apache on each host i have a VM acting as a reverse proxy.
So pfsense has port 80 forwarded to the proxy then apache setup as a reverse proxy for the internal boxes, it decides which VM it goes to by either a CNAME or the directory (example.com/cacti).

gregoryfenton
04-10-2013, 11:41
My setup:
KS2013 host
5 VMs running on the host, 192.168.0.200-204
192.168.0.200 is a domain name server and a http proxy server forwarding http requests to the individual VMs based on the domain name

So for example if someone asked for http://vm1.mydomain.com the proxy server would forward the request to 192.168.0.201.

Each server can be connected to directly using port forwarding so connecting to port 2200 would be the ssh port (22) of the proxy VM, 2201 would be the ssh port of VM1, 2202 -> VM2, 2203 -> VM3, 2204 -> VM4 etc and the same with http (80) 8000 -> proxy, 8001 -> VM1, 8002 -> VM2, 8003 -> VM3, 8004 -> VM5

If a VM does not provide a service on the particular port the request will timeout and return a standard port unreachable response.

Code:
#!/bin/bash
sys=`which sysctl`
ipt=`which iptables`
mp=`which modprobe`
grep=`which grep`

if [[ "$sys" == "" || $ipt == "" || $mp == "" || $grep == "" ]]; then
  echo "Error - needed program not found"
  return -1
fi

services=/etc/services

function getport
{
  local a b p
  p=$1
  if [ "$p" == "" ]; then
    echo "noport"
    return
  fi
  a=`$grep -m 1 -w "$p" $services`
  if [ "$a" == "" ]; then
    echo "no match"
    return
  fi
  b=${a%%/*}
  p=${b%%$p*}
  echo -n $p
}

ipprefix=192.168.0

#the IP below ($proxyip) handles all requests for $proxiedports from the outside world
proxyip=200
#Starting IP
firstip=200

dns=53
http=80
ftp=21
ssh=22
rsync=873
proxiedports=( $dns $http )

#VM names
vm1=201
vm2=202
vm3=203
vm4=204
toips=( $proxyip $vm1 $vm2 $vm3 $vm4 )
ports=( $ftp $ssh $dns $http $rsync )

netmask=24
extif=vmbr0
intif=vmbr1

#Flush iptables
$ipt -F
$ipt -t nat -F

# Net Sharing
$mp iptable_nat

echo Enabling ipv4 forwarding
$sys net.ipv4.ip_forward=1
echo 1 > /proc/sys/net/ipv4/ip_forward

$ipt -t nat -A POSTROUTING -o eth0 -j MASQUERADE
$ipt -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
$ipt -A INPUT -i lo -j ACCEPT
$ipt -A INPUT -j LOG --log-level 4 --log-prefix "ATTACK"
$ipt -A INPUT -j DROP

# Add your additional rules here
$ipt -t nat -A POSTROUTING -s $ipprefix.0/$netmask -j MASQUERADE

#hardcoded rules
#default services that need remapping and must remain on a standard port:
# DNS (port $dns)
# Web server (port $http)
#both of these will be handled by the first VM

for port in ${proxiedports[@]}; do
echo Setting up $ipprefix.$proxyip to handle external calls to port $port \($(getport $port)\)
  $ipt -t nat -A PREROUTING -p tcp  -i $extif --dport $port -j DNAT --to-destination $ipprefix.$proxyip:$port
  $ipt -t nat -A POSTROUTING -p tcp -o $extif -d $ipprefix.$proxyip --dport $port -j MASQUERADE
  $ipt -t nat -A PREROUTING -p udp -i $extif --dport $port -j DNAT --to-destination $ipprefix.$proxyip:$port
  $ipt -t nat -A POSTROUTING -p udp -o $extif -d $ipprefix.$proxyip --dport $port -j MASQUERADE
done

count=0
for toip in ${toips[@]}; do
  echo Configuring VM $ipprefix.$toip
  for port in ${ports[@]}; do
    oldport=$port
    if [ $(printf "%.0f" $port) -gt 99 ]; then
      port=$(echo "scale=1; $port / 10" | bc)
      if [ $(printf "%.0f" $port) -gt 99 ]; then
        port=$(echo "scale=1; $port / 10" | bc)
      fi
    fi

    mport=$(printf "%.0f" `echo "scale=0; ($port * 100) + ($toip - $firstip)" | bc`)
    port=$oldport
    echo mapping external port $mport to $ipprefix.$toip:$port \($(getport $port)\)
    $ipt -t nat -D PREROUTING -i $extif -p tcp --dport $mport -j DNAT --to-destination $ipprefix.$toip:$port > /dev/null 2>&1
    $ipt -t nat -D POSTROUTING -o $extif -p tcp -d $ipprefix.$toip --dport $port -j MASQUERADE > /dev/null 2>&1
    $ipt -t nat -A PREROUTING -i $extif -p tcp --dport $mport -j DNAT --to-destination $ipprefix.$toip:$port
    $ipt -t nat -A POSTROUTING -o $extif -p tcp -d $ipprefix.$toip --dport $port -j MASQUERADE
  done
done
return 0
And here is the output:
Code:
Enabling ipv4 forwarding
net.ipv4.ip_forward = 1
Setting up 192.168.0.200 to handle external calls to port 53 (domain)
Setting up 192.168.0.200 to handle external calls to port 80 (http)
Configuring VM 192.168.0.200
mapping external port 2100 to 192.168.0.200:21 (ftp)
mapping external port 2200 to 192.168.0.200:22 (ssh)
mapping external port 5300 to 192.168.0.200:53 (domain)
mapping external port 8000 to 192.168.0.200:80 (http)
mapping external port 8730 to 192.168.0.200:873 (rsync)
Configuring VM 192.168.0.201
mapping external port 2101 to 192.168.0.201:21 (ftp)
mapping external port 2201 to 192.168.0.201:22 (ssh)
mapping external port 5301 to 192.168.0.201:53 (domain)
mapping external port 8001 to 192.168.0.201:80 (http)
mapping external port 8731 to 192.168.0.201:873 (rsync)
Configuring VM 192.168.0.202
mapping external port 2102 to 192.168.0.202:21 (ftp)
mapping external port 2202 to 192.168.0.202:22 (ssh)
mapping external port 5302 to 192.168.0.202:53 (domain)
mapping external port 8002 to 192.168.0.202:80 (http)
mapping external port 8732 to 192.168.0.202:873 (rsync)
Configuring VM 192.168.0.203
mapping external port 2103 to 192.168.0.203:21 (ftp)
mapping external port 2203 to 192.168.0.203:22 (ssh)
mapping external port 5303 to 192.168.0.203:53 (domain)
mapping external port 8003 to 192.168.0.203:80 (http)
mapping external port 8733 to 192.168.0.203:873 (rsync)
Configuring VM 192.168.0.204
mapping external port 2104 to 192.168.0.204:21 (ftp)
mapping external port 2204 to 192.168.0.204:22 (ssh)
mapping external port 5304 to 192.168.0.204:53 (domain)
mapping external port 8004 to 192.168.0.204:80 (http)
mapping external port 8734 to 192.168.0.204:873 (rsync)