OVH Community, your new community space.

Help request, iptables, VMs


gregoryfenton
04-10-2013, 11:23
After a (definitely earned) good night's sleep I have completed my script.

Thanks for all your help Myatu, it was massively appreciated.

I have added this script to the howto section, save it getting lost
http://forum.ovh.co.uk/showthread.php?p=55961#post55961

Here's my script:
Code:
#!/bin/bash
sys=`which sysctl`
ipt=`which iptables`
mp=`which modprobe`
grep=`which grep`

if [[ "$sys" == "" || $ipt == "" || $mp == "" || $grep == "" ]]; then
  echo "Error - needed program not found"
  return -1
fi

services=/etc/services

function getport
{
  local a b p
  p=$1
  if [ "$p" == "" ]; then
    echo "noport"
    return
  fi
  a=`$grep -m 1 -w "$p" $services`
  if [ "$a" == "" ]; then
    echo "no match"
    return
  fi
  b=${a%%/*}
  p=${b%%$p*}
  echo -n $p
}

ipprefix=192.168.0

#the IP below ($proxyip) handles all requests for $proxiedports from the outside world
proxyip=200
#Starting IP
firstip=200

dns=53
http=80
ftp=21
ssh=22
rsync=873
proxiedports=( $dns $http )

#VM names
vm1=201
vm2=202
vm3=203
vm4=204
toips=( $proxyip $vm1 $vm2 $vm3 $vm4 )
ports=( $ftp $ssh $dns $http $rsync )

netmask=24
extif=vmbr0
intif=vmbr1

#Flush iptables
$ipt -F
$ipt -t nat -F

# Net Sharing
$mp iptable_nat

echo Enabling ipv4 forwarding
$sys net.ipv4.ip_forward=1
echo 1 > /proc/sys/net/ipv4/ip_forward

$ipt -t nat -A POSTROUTING -o eth0 -j MASQUERADE
$ipt -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
$ipt -A INPUT -i lo -j ACCEPT
$ipt -A INPUT -j LOG --log-level 4 --log-prefix "ATTACK"
$ipt -A INPUT -j DROP

# Add your additional rules here
$ipt -t nat -A POSTROUTING -s $ipprefix.0/$netmask -j MASQUERADE

#hardcoded rules
#default services that need remapping and must remain on a standard port:
# DNS (port $dns)
# Web server (port $http)
#both of these will be handled by the first VM

for port in ${proxiedports[@]}; do
echo Setting up $ipprefix.$proxyip to handle external calls to port $port \($(getport $port)\)
  $ipt -t nat -A PREROUTING -p tcp  -i $extif --dport $port -j DNAT --to-destination $ipprefix.$proxyip:$port
  $ipt -t nat -A POSTROUTING -p tcp -o $extif -d $ipprefix.$proxyip --dport $port -j MASQUERADE
  $ipt -t nat -A PREROUTING -p udp -i $extif --dport $port -j DNAT --to-destination $ipprefix.$proxyip:$port
  $ipt -t nat -A POSTROUTING -p udp -o $extif -d $ipprefix.$proxyip --dport $port -j MASQUERADE
done

count=0
for toip in ${toips[@]}; do
  echo Configuring VM $ipprefix.$toip
  for port in ${ports[@]}; do
    oldport=$port
    if [ $(printf "%.0f" $port) -gt 99 ]; then
      port=$(echo "scale=1; $port / 10" | bc)
      if [ $(printf "%.0f" $port) -gt 99 ]; then
        port=$(echo "scale=1; $port / 10" | bc)
      fi
    fi

    mport=$(printf "%.0f" `echo "scale=0; ($port * 100) + ($toip - $firstip)" | bc`)
    port=$oldport
    echo mapping external port $mport to $ipprefix.$toip:$port \($(getport $port)\)
    $ipt -t nat -D PREROUTING -i $extif -p tcp --dport $mport -j DNAT --to-destination $ipprefix.$toip:$port > /dev/null 2>&1
    $ipt -t nat -D POSTROUTING -o $extif -p tcp -d $ipprefix.$toip --dport $port -j MASQUERADE > /dev/null 2>&1
    $ipt -t nat -A PREROUTING -i $extif -p tcp --dport $mport -j DNAT --to-destination $ipprefix.$toip:$port
    $ipt -t nat -A POSTROUTING -o $extif -p tcp -d $ipprefix.$toip --dport $port -j MASQUERADE
  done
done
return 1

gregoryfenton
04-10-2013, 00:07
Much appreciated. I knew I was close, and it clicked in my head from your first reply.

Thanks for the DNS heads up

All this hard work just so I can have an easy life :P

Night Myatu

Myatu
04-10-2013, 00:03
Quote Originally Posted by gregoryfenton
Would that do it?
Likely Leave the POSTROUTING bits for these ports out though - just one POSTROUTING, and that's for ouptut on vmbr0 if source is 192.yada

Myatu
04-10-2013, 00:02
I don't see a specific interface on the "PREROUTING" entries for NAT ("-t nat"). Should be something like:

Code:
-A PREROUTING -i vmbr0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.200
The "-i vmbr0" is the important part. That should be something you could add to the script.

You can add to that:

Code:
-A OUTPUT -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.2.200
PS: The MASQUERADE should only be needed once, ie:

Code:
-A POSTROUTING -o vmbr0 -s 192.168.0.0/23 -j MASQUERADE
P.P.S: DNS uses UDP 53 by default (TCP 53 is a lesser supported one).

gregoryfenton
03-10-2013, 23:53
Ahhh I see what you're saying.

If I change the anywhere:80 and anywhere:53 section from
Code:
for port in $(echo 53 80); do
  $ipt -t nat -D PREROUTING -p tcp --dport $port -j DNAT --to-destination $ipprefix$toip:$port > /dev/null 2>&1
  $ipt -t nat -D POSTROUTING -p tcp -d $ipprefix$toip --dport $port -j MASQUERADE > /dev/null 2>&1
  $ipt -t nat -A PREROUTING -p tcp --dport $port -j DNAT --to-destination $ipprefix$toip:$port
  $ipt -t nat -A POSTROUTING -p tcp -d $ipprefix$toip --dport $port -j MASQUERADE
done
to
Code:
for port in $(echo 53 80); do
  $ipt -t nat -D PREROUTING -p tcp -i vmbr0 --dport $port -j DNAT --to-destination $ipprefix$toip:$port > /dev/null 2>&1
  $ipt -t nat -D POSTROUTING -p tcp -i vmbr0  -d $ipprefix$toip --dport $port -j MASQUERADE > /dev/null 2>&1
  $ipt -t nat -A PREROUTING -p tcp -i vmbr0  --dport $port -j DNAT --to-destination $ipprefix$toip:$port
  $ipt -t nat -A POSTROUTING -p tcp -i vmbr0  -d $ipprefix$toip --dport $port -j MASQUERADE
done
Would that do it?

gregoryfenton
03-10-2013, 23:48
iptables -S
Code:
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A INPUT -m state --state NEW -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
iptables -t nat -S
Code:
-P PREROUTING ACCEPT
-P POSTROUTING ACCEPT
-P OUTPUT ACCEPT
-A PREROUTING -p tcp -m tcp --dport 53 -j DNAT --to-destination 192.168.0.200:53
-A PREROUTING -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.0.200:80
-A PREROUTING -p tcp -m tcp --dport 2100 -j DNAT --to-destination 192.168.0.200:21
-A PREROUTING -p tcp -m tcp --dport 2200 -j DNAT --to-destination 192.168.0.200:22
-A PREROUTING -p tcp -m tcp --dport 5300 -j DNAT --to-destination 192.168.0.200:53
-A PREROUTING -p tcp -m tcp --dport 8000 -j DNAT --to-destination 192.168.0.200:80
-A PREROUTING -p tcp -m tcp --dport 8730 -j DNAT --to-destination 192.168.0.200:873
-A PREROUTING -p tcp -m tcp --dport 2101 -j DNAT --to-destination 192.168.0.201:21
-A PREROUTING -p tcp -m tcp --dport 2201 -j DNAT --to-destination 192.168.0.201:22
-A PREROUTING -p tcp -m tcp --dport 5301 -j DNAT --to-destination 192.168.0.201:53
-A PREROUTING -p tcp -m tcp --dport 8001 -j DNAT --to-destination 192.168.0.201:80
-A PREROUTING -p tcp -m tcp --dport 8731 -j DNAT --to-destination 192.168.0.201:873
-A PREROUTING -p tcp -m tcp --dport 2102 -j DNAT --to-destination 192.168.0.202:21
-A PREROUTING -p tcp -m tcp --dport 2202 -j DNAT --to-destination 192.168.0.202:22
-A PREROUTING -p tcp -m tcp --dport 5302 -j DNAT --to-destination 192.168.0.202:53
-A PREROUTING -p tcp -m tcp --dport 8002 -j DNAT --to-destination 192.168.0.202:80
-A PREROUTING -p tcp -m tcp --dport 8732 -j DNAT --to-destination 192.168.0.202:873
-A PREROUTING -p tcp -m tcp --dport 2103 -j DNAT --to-destination 192.168.0.203:21
-A PREROUTING -p tcp -m tcp --dport 2203 -j DNAT --to-destination 192.168.0.203:22
-A PREROUTING -p tcp -m tcp --dport 5303 -j DNAT --to-destination 192.168.0.203:53
-A PREROUTING -p tcp -m tcp --dport 8003 -j DNAT --to-destination 192.168.0.203:80
-A PREROUTING -p tcp -m tcp --dport 8733 -j DNAT --to-destination 192.168.0.203:873
-A PREROUTING -p tcp -m tcp --dport 2104 -j DNAT --to-destination 192.168.0.204:21
-A PREROUTING -p tcp -m tcp --dport 2204 -j DNAT --to-destination 192.168.0.204:22
-A PREROUTING -p tcp -m tcp --dport 5304 -j DNAT --to-destination 192.168.0.204:53
-A PREROUTING -p tcp -m tcp --dport 8004 -j DNAT --to-destination 192.168.0.204:80
-A PREROUTING -p tcp -m tcp --dport 8734 -j DNAT --to-destination 192.168.0.204:873
-A POSTROUTING -s 192.168.0.0/23 -o vmbr0 -j MASQUERADE
-A POSTROUTING -s 192.168.0.0/23 -o vmbr1 -j MASQUERADE
-A POSTROUTING -d 192.168.0.200/32 -p tcp -m tcp --dport 21 -j MASQUERADE
-A POSTROUTING -d 192.168.0.200/32 -p tcp -m tcp --dport 22 -j MASQUERADE
-A POSTROUTING -d 192.168.0.200/32 -p tcp -m tcp --dport 53 -j MASQUERADE
-A POSTROUTING -d 192.168.0.200/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -d 192.168.0.200/32 -p tcp -m tcp --dport 873 -j MASQUERADE
-A POSTROUTING -d 192.168.0.201/32 -p tcp -m tcp --dport 21 -j MASQUERADE
-A POSTROUTING -d 192.168.0.201/32 -p tcp -m tcp --dport 22 -j MASQUERADE
-A POSTROUTING -d 192.168.0.201/32 -p tcp -m tcp --dport 53 -j MASQUERADE
-A POSTROUTING -d 192.168.0.201/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -d 192.168.0.201/32 -p tcp -m tcp --dport 873 -j MASQUERADE
-A POSTROUTING -d 192.168.0.202/32 -p tcp -m tcp --dport 21 -j MASQUERADE
-A POSTROUTING -d 192.168.0.202/32 -p tcp -m tcp --dport 22 -j MASQUERADE
-A POSTROUTING -d 192.168.0.202/32 -p tcp -m tcp --dport 53 -j MASQUERADE
-A POSTROUTING -d 192.168.0.202/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -d 192.168.0.202/32 -p tcp -m tcp --dport 873 -j MASQUERADE
-A POSTROUTING -d 192.168.0.203/32 -p tcp -m tcp --dport 21 -j MASQUERADE
-A POSTROUTING -d 192.168.0.203/32 -p tcp -m tcp --dport 22 -j MASQUERADE
-A POSTROUTING -d 192.168.0.203/32 -p tcp -m tcp --dport 53 -j MASQUERADE
-A POSTROUTING -d 192.168.0.203/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -d 192.168.0.203/32 -p tcp -m tcp --dport 873 -j MASQUERADE
-A POSTROUTING -d 192.168.0.204/32 -p tcp -m tcp --dport 21 -j MASQUERADE
-A POSTROUTING -d 192.168.0.204/32 -p tcp -m tcp --dport 22 -j MASQUERADE
-A POSTROUTING -d 192.168.0.204/32 -p tcp -m tcp --dport 53 -j MASQUERADE
-A POSTROUTING -d 192.168.0.204/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -d 192.168.0.204/32 -p tcp -m tcp --dport 873 -j MASQUERADE

Myatu
03-10-2013, 23:42
What's the output of "iptables -S" and "iptables -t nat -S" ? From the sounds of it, its a bit of overzealous port forwarding in the wrong direction (from "anything" to port 80, forward to 192.168.1.200, instead of from "vmbr0").

gregoryfenton
03-10-2013, 19:22
Hi all

I bit the bullet and made my server into several proxmox VMs.

Here's the layout:

host: 188.x.y.z
APSVM: 192.168.1.200 (apache reverse proxy server)
VM1-ME: 192.168.1.201 (my stuff)
VM2-JIM: 192.168.1.202 (a friend's VM)
VM3-JAIL: 192.168.1.203 (jailed VM)
VM4-MIRROR: 192.168.1.204 (my mirror)

I have got port forwarding working so connections to the host from the internet go to the correct VM.

My issue is that any http request I make from a VM gets the webpage back from APSVM rather than the actual site. I can ping the site and get the correct IP address.

Any ideas?

Below is the script I use:
Code:
#!/bin/bash
sys=`which sysctl`
ipt=`which iptables`

$ipt -t nat -F

#hardcoded rules
#default services that need remapping and must remain on a standard port:
# DNS (port 53)
# Web server (port 80)
#both of these will be handled by the first VM

ipprefix=192.168.0.
toip=200

iptables -D INPUT  -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -D OUTPUT -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT

iptables -A INPUT  -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT

$ipt -t nat -D POSTROUTING -s 192.168.0.0/23 -o vmbr0 -j MASQUERADE > /dev/null 2>&1
$ipt -t nat -A POSTROUTING -s 192.168.0.0/23 -o vmbr0 -j MASQUERADE
$ipt -t nat -D POSTROUTING -s 192.168.0.0/23 -o vmbr1 -j MASQUERADE > /dev/null 2>&1
$ipt -t nat -A POSTROUTING -s 192.168.0.0/23 -o vmbr1 -j MASQUERADE
for port in $(echo 53 80); do
  $ipt -t nat -D PREROUTING -p tcp --dport $port -j DNAT --to-destination $ipprefix$toip:$port > /dev/null 2>&1
  $ipt -t nat -D POSTROUTING -p tcp -d $ipprefix$toip --dport $port -j MASQUERADE > /dev/null 2>&1
  $ipt -t nat -A PREROUTING -p tcp --dport $port -j DNAT --to-destination $ipprefix$toip:$port
  $ipt -t nat -A POSTROUTING -p tcp -d $ipprefix$toip --dport $port -j MASQUERADE
done

toips=( 200 201 202 203 204 )
ports=( 21 22 53 80 873 )

$sys net.ipv4.ip_forward=1

count=0
for toip in ${toips[@]}; do
  for port in ${ports[@]}; do
    oldport=$port
    if [ $(printf "%.0f" $port) -gt 99 ]; then
      port=$(echo "scale=1; $port / 10" | bc)
      if [ $(printf "%.0f" $port) -gt 99 ]; then
        port=$(echo "scale=1; $port / 10" | bc)
      fi
    fi

    mport=$(printf "%.0f" `echo "scale=0; ($port * 100) + ($toip - 200)" | bc`)
    port=$oldport
    echo mapping local port $mport to $ipprefix$toip:$port
    $ipt -t nat -D PREROUTING -p tcp --dport $mport -j DNAT --to-destination $ipprefix$toip:$port > /dev/null 2>&1
    $ipt -t nat -D POSTROUTING -p tcp -d $ipprefix$toip --dport $port -j MASQUERADE > /dev/null 2>&1
    $ipt -t nat -A PREROUTING -p tcp --dport $mport -j DNAT --to-destination $ipprefix$toip:$port
    $ipt -t nat -A POSTROUTING -p tcp -d $ipprefix$toip --dport $port -j MASQUERADE
  done
done