1 2 3 4
How to convert a Virtual PC 2007 VMC file to work with Hyper-V

Windows 8 was made publicly available a few weeks ago. As usual, Microsoft manages to guarantee compatibility with a lot of software, but there are a few tools that will simply no longer run.

One of these is Microsoft's own Security Essentials product, which has been completely replaced with Windows Defender, which is built right in to Windows 8. So that one's easy.

Another is Microsoft Virtual PC 2007. It doesn't run under Windows 8 at all. Neither is the configuration format that it uses directly compatible with any of the other virtualization solutions that do run under Windows 8.

  • As of November 2012, VirtualBox is still having some compatibility and speed problems under Windows 8
  • VMWare's runner also doesn't have an easy upgrade path for Virtual PC images. You have to convert the disk image and somehow recreate the VM configuration file
  • Even Microsoft's own Hyper-V is only available on machines that have hardware support for it and, while the disk image is compatible, the configuration format is completely different

If you're already a user of Microsoft's Virtual PC, then it's likely you'd like to just upgrade to using Hyper-V, if possible. Luckily, Hyper-V is available as an option for Windows 8 Pro and higher. To find out if your machine supports it and to install it, follow the instructions below.

Enable Hyper-V

image

  • Press Windows key+W to search settings
  • Type "win fea" and Enter to show the "Turn Windows Features On and Off" window
  • If the "Hyper-V" checkbox is already checked, then you're ready for the next step
  • If the "Hyper-V" checkbox is disabled, then you're out of luck; Hyper-V is not available for your machine and you'll have to try one of the other virtualization solutions mentioned above
  • Otherwise, check the "Hyper-V" checkbox and press "Ok". You'll naturally have to reboot for those changes to be applied.

Configure the Hyper-V Switch

Once Hyper-V is enabled and you've rebooted, you can startup the Hyper-V Manager and configure it.

  • Press the Windows key to show the start screen
  • Type "hyper" and Enter to find and start the "Hyper-V Manager"
  • Select your machine in the tree on the left under Hyper-V Manager
  • In the settings for that machine on the right, click "Virtual Switch Manager"
  • In that dialog, the "New virtual network switch" node should be selected and you'll see a list on the right. Select "External" to create a switch that has access to the Internet and press "Create Virtual Switch"

imageimageimage

At this point, your Hyper-V server is ready to load your virtual machine and let it access the Internet.

Create a Hyper-V virtual machine from a .vmc file

All of the configuration settings for the Virtual PC virtual machine are stored in a .VMC file. Unfortunately, the Hyper-V manager can't import these files directly1. Luckily, there is a tool, called the VMC to Hyper-V Import Tool, which performs the import in a couple of easy steps.

  • Download, extract, install and run the tool
  • First, press "Connect" to attach to the local Hyper-V instance (if you just installed, then there is no user and password set)
  • Next, open the .vmc file for the virtual machine you want to import
  • The settings are loaded into the window; verify that they more-or-less match what you expect.2
  • Press the "Create Virtual Machine" button to create a new virtual machine in Hyper-V based on those settings.

imageimageimage

You're now ready to configure and start up your virtual machine.

Configuring and Running the VM

There are two things to do to get this machine running smoothly under Hyper-V:

  • Set up the network interface
  • Install the Integration Services, which includes drivers but essentially makes the mouse work as expected and enables non-legacy networking for guest OSs that support it

There are two kinds of network interface: the standard one and a legacy one. If your guest operating system is Windows XP (as mine was), you have to use the legacy adapter. The documentation also says that a legacy adapter is required to have connectivity without the "Integration Services".3

Install a Legacy Adapter

If you have Windows XP, you can just remove the "Network Adapter" that's already included and instead install a "Legacy Adapter".

  • From the Hyper-V Manager, select the settings for your machine
  • "Add Hardware" is automatically selected
  • Select the "Legacy Network Adapter" and press "Add"
  • Select the existing "Network Adapter" in the list on the left to show its settings
  • Press the "Remove" button to remove the unneeded "Network Adapter"

imageimageimage

You can now set up the network for that adapter, as shown below.

Set up the network

  • From the Hyper-V Manager, select the settings for your machine
  • Select the "Legacy Network Adapter" or "Network Adapter" to show its settings
  • Assign the switch (created in a step above) to the network and press OK to save settings

imageimageimage

At this point, your virtual machine should be able to connect to the network once it's started.

Install Integration Services

The machine is not very useful until you've installed the integration services. These services enable seamless mouse support and also enable networking over a non-legacy adapter.

image

  • Start the VM
  • Wait for the machine to finish booting4
  • You can't install the integration services until the previous integration tools have been uninstalled. If your guest OS is Windows XP, uninstall the Virtual PC tools using the "Add/Remove Programs" control panel
  • Once all other tools are uninstalled, select "Insert Integration Services Setup Disk" from the "Action" menu
  • After a few seconds, the installation should start automatically
  • You'll have to reboot the guest OS to finish installation.

That's it! Your Windows XP should once again have full hardware support, including legacy networking (up to 100Mb). Adjust your display settings back up to a usable resolution, re-activate with Microsoft (you have three days) and enjoy your new Hyper-V virtual machine.



  1. It's an utter mystery why Microsoft couldn't be bothered to provide an upgrade path from its own product. Perhaps they didn't want to "officially" support such upgrades in order to kill off as many virtual machines running Windows XP as possible.

  2. imageIn my case, the path to the main disk image was incorrect and showed up in red. It's a mystery why that file had such an old path in it, while the VM started with the correct disk image in Virtual PC. At any rate, I adjusted the path to point to the correct disk image, the text turned black and I was allowed to continue.

  3. It's unclear to me whether network connectivity is required in order to install the integration tools. It took several attempts before the integration services installed successfully. It's possible that this was due to the unsatisfactory network situation, but I can't say for sure.

  4. And, if you're using Windows XP as the guest OS, until it has stopped complaining about hardware changes and activation problems

How to configure a local firewall for OpenVPN (Part II)

The following tip was developed using Ubuntu 9.1x (Hardy Heron) with OpenVPn 2.1rc19. It builds on the the setup from Part I.

This post has been updated since publication to include FORWARD directives for the strangers list as well.


Part I of this guide to configuring a local firewall for OpenVPN introduced you to using iptables on Linux. It also included a script for OpenVPN that opened and closed the firewall for specific IP addresses. If you haven't read it already, you should probably go do that first.

Unfortunately, it turns out that the firewall configuration from part I is not watertight because it still allows FORWARDs for all IP addresses. If you'll recall, we solved this problem for INPUTs by closing them by default and selectively opening them.

The first step is to ascertain that the firewall is configured as we expect. A call to sudo iptables -nL elicits the following output:

Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

As you can see, the default policy for FORWARD is ACCEPT, which allows anyone to access other IP addresses from this machine. In Part I, you created a file named /etc/iptables.uprules in which you stored the default configuration of the firewall. You'll want to change that as shown below (the changes are highlighted):

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i eth0 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A FORWARD -i eth0 -j ACCEPT
-A FORWARD -i lo -j ACCEPT
COMMIT

Restart networking by executing sudo /etc/init.d/networking restart. A call to sudo iptables -nL should now elicit the following output (the main changes are highlighted):

Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Now all IP forwarding requests are blocked by default. Those for the lo and eth0 interfaces are, of course, still enabled, to allow the machine to be reachable both by itself and the local network.

The final step is to change the firewall configuration script to open up IP forwarding for employees, but not for strangers. Since this is a FORWARD rule, not an INPUT one, the script has to make sure the remove all firewall rules for the client IP address instead of just the INPUT rules as it did previously. Changes from the script in Part I are highlighted.

function inlist
{
  less `dirname $0`/$1 | egrep "^${CLIENTCERT}$" > /dev/null
  if [ $? -eq 0 ]; then
    return 0
  else
    return 1
  fi
}

function get_next_matching_firewall_rule
{
  ip_address=$1
  channel=$2

  RULE="`iptables -L $channel -n --line-numbers | grep $ip_address | head -n 1`"
}

function drop_rule_from_iptables
{
  rule="$1"
  channel="$2"
  echo "  Drop rule [$rule] for channel [$channel]"
  line_number=`echo "$rule" | awk '{print $1}'`
  iptables -D $channel $line_number
}

function add_port_to_iptables
{
  source_ip=$1
  destination_ip=$2
  protocol=$3
  port=$4

  iptables -A INPUT -i tun0 -s $source_ip -d $destination_ip -p $protocol --dport $port -j ACCEPT
  iptables -A INPUT -i tun0 -s $source_ip -d $destination_ip -p $protocol --dport $port -j ACCEPT
}

function add_destination_to_iptables
{
  source_ip=$1
  destination_ip=$2
  iptables -A INPUT -i tun0 -s $source_ip -d $destination_ip -j ACCEPT
  iptables -A FORWARD -i tun0 -s $source_ip -d $destination_ip -j ACCEPT
}

function open_firewall_for_strangers
{
  echo "  Add route for DNS"
  add_port_to_iptables $CLIENTIP 192.168.1.1 "UDP" 53

  echo "  Add route for Windows shares"
  add_port_to_iptables $CLIENTIP 192.168.1.5 "TCP" 139
  add_port_to_iptables $CLIENTIP 192.168.1.5 "TCP" 445

  return 0
}

function open_firewall_for_employees
{
  echo "  Add routes for all ip addresses"
  iptables -A INPUT -i tun0 -s $CLIENTIP -j ACCEPT
  iptables -A FORWARD -i tun0 -s $CLIENTIP -j ACCEPT
  return 0
}

function open_firewall
{
  echo "Opening firewall for $CLIENTCERT @ [$CLIENTIP]"
# TODO Add filtering for other lists, if desired
# inlist "MYGROUP.list"
#if [ $? -eq 0 ]; then
#  echo "  Certificate found in MYGROUP list"
#  open_firewall_for_MYGROUP
#  return 0
#else
  inlist "strangers.list"
  if [ $? -eq 0 ]; then
    echo "  Certificate found in strangers list"
    open_firewall_for_strangers
    return 0
  else
    inlist "employees.list"
    if [ $? -eq 0 ]; then
      echo "  Certificate found in employee list"
      open_firewall_for_employees
      return 0
    else
      echo "  Certificate not found in any list"
      return 1
    fi
  fi
}

function close_firewall_channel
{
  channel=$1

  get_next_matching_firewall_rule $CLIENTIP $channel

  while [ -n "$RULE" ]
  do
    drop_rule_from_iptables "$RULE" $channel
    get_next_matching_firewall_rule $CLIENTIP $channel
  done

}

function close_firewall
{
  echo "CloseFirewall for [$CLIENTIP]"

  close_firewall_channel "INPUT"
  close_firewall_channel "FORWARD"
  close_firewall_channel "OUTPUT"
}

# Main

OPERATION=$1
CLIENTIP=$2
CLIENTCERT=$3

case "$1" in
  add)
    close_firewall
    open_firewall
    ;;

  update)
    close_firewall
    open_firewall
    ;;

  delete)
    close_firewall
    ;;
  *)
    echo "Unknown operation"
    exit 1
esac

exit $?

Since you only changed the firewall configuration script, there is no need to restart OpenVPN.

Testing the Script

You can test to verify that the firewall is updated properly by simply executing the /etc/openvpn/configfirewall.sh script with various parameters. The expected parameters are an operation -- "add" or "delete" for testing purposes -- a name -- matched against the names in your lists -- and an IP address, which should be chosen so as not to interfere with any addresses assigned by either OpenVPN or a DHCP server.

To test what would happen when an employee connects through OpenVPN, execute the following command:

sudo /etc/openvpn/configfirewall.sh add 192.168.40.3 John_Doe

You should see the following output from the script:

CloseFirewall for [192.168.40.3]
OpenFirewall for John_Doe @ [192.168.40.3]
  Certificate found in employee list
  Add routes for all ip addresses

This sounds about right and it looks like the script ran as expected. You can check that the firewall was configured as expected with a call to sudo iptables -nL, which should now elicit the following output (the main changes are highlighted):

Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  192.168.40.3         0.0.0.0/0

Chain FORWARD (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  192.168.40.3         0.0.0.0/0

As you can see, the firewall accepts all INPUT and FORWARD from employees. Removing this test employee is as simple as executing:

sudo /etc/openvpn/configfirewall.sh delete 192.168.40.3 John_Doe

You should see the following output from the script:

CloseFirewall for [192.168.40.3]
  Drop rule [4    ACCEPT     all  --  192.168.40.3         0.0.0.0/0           ] for channel [INPUT]
  Drop rule [4    ACCEPT     all  --  192.168.40.3         0.0.0.0/0           ] for channel [FORWARD]

A call to sudo iptables -nL should now elicit the following output, where the rules for the employee have been removed:

Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

You should really test with one user from each list, so the next user to test is a stranger. Add a stranger by calling the script with the strangers's name instead of the employee's name:

sudo /etc/openvpn/configfirewall.sh add 192.168.40.3 John_Stranger

You should see the following output from the script:

CloseFirewall for [192.168.40.3]
OpenFirewall for John_Stranger @ [192.168.40.3]
  Certificate found in strangers list
  Add route for DNS
  Add route for Windows shares

A call to sudo iptables -nL should now elicit the following output:

Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     udp  --  192.168.40.3         192.168.1.1       udp dpt:53
ACCEPT     tcp  --  192.168.40.3         192.168.1.5       tcp dpt:139
ACCEPT     tcp  --  192.168.40.3         192.168.1.5       tcp dpt:445

Chain FORWARD (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     udp  --  192.168.40.3         192.168.1.1       udp dpt:53
ACCEPT     tcp  --  192.168.40.3         192.168.1.5       tcp dpt:139
ACCEPT     tcp  --  192.168.40.3         192.168.1.5       tcp dpt:445

For strangers, the firewall accepts only requests on the ports and IP addresses explicitly opened by the script and drops all FORWARD requests. Removing this test employee is as simple as executing:

sudo /etc/openvpn/configfirewall.sh delete 192.168.40.3 John_Stranger

You should see the following output from the script:

CloseFirewall for [192.168.40.3]
  Drop rule [4    ACCEPT     udp  --  192.168.40.3         192.168.1.1       udp dpt:53 ] for channel [INPUT]
  Drop rule [4    ACCEPT     tcp  --  192.168.40.3         192.168.1.5       tcp dpt:139 ] for channel [INPUT]
  Drop rule [4    ACCEPT     tcp  --  192.168.40.3         192.168.1.5       tcp dpt:445 ] for channel [INPUT]
  Drop rule [4    ACCEPT     udp  --  192.168.40.3         192.168.1.1       udp dpt:53 ] for channel [FORWARD]
  Drop rule [4    ACCEPT     tcp  --  192.168.40.3         192.168.1.5       tcp dpt:139 ] for channel [FORWARD]
  Drop rule [4    ACCEPT     tcp  --  192.168.40.3         192.168.1.5       tcp dpt:445 ] for channel [FORWARD]

A call to sudo iptables -nL should now elicit the following output, where the rules for the stranger have been removed:

Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

You can use the script this way to test the firewall configuration without actually logging in through OpenVPN. When everything is set, you should still log with OpenVPN as a user from each list to verify that the firewall is doing what you think it is doing. In fact, that's exactly why there is a Part II to this article: We tested by adding a user to the strangers list and logging in and noticed that we were able to ping many more servers than we had configured. Don't let that happen to you!

Testing via OpenVPN

So, there's one more trick that you can use to make testing via OpenVPN easier. Since you have to be outside the network to test tunneling in via VPN, you run into the problem of testing as a stranger because strangers probably won't have rights to open a shell on the OpenVPN server. That is, you need to be able to do this:

  1. Log in with VPN as an employee
  2. Add yourself to the strangers list
  3. Log out of VPN
  4. Log in with VPN as a stranger
  5. Test that you cannot access anything that you shouldn't be able to
  6. remove yourself from the strangers list (uh oh!)

Since you're a stranger, you can no longer open a shell on the OpenVPN server and alter the configuration.

Here are some ways of getting around this problem:

  • The safest and best way to test via OpenVPN is to create a user exclusively for testing and add that user to each of your lists in turn, logging in and out to test the configuration. If you need to change something, you just log out and log back in using your "employee" user.
  • Another way is to open an OpenVPN session from one machine (it can be a virtual machine) and then add that same user to each of your lists in turn, logging in and out from another machine. Just make sure not to lose the connection from the "main" machine or you may be locked out.
  • Another way around this is to add an exception for the OpenVPN server to all configurations (strangers, employees, etc.) so that you can test almost everything. To do this, just add the a rule for the OpenVPN server (assumed to be on 192.168.1.1) as follows:
add_destination_to_iptables $CLIENTIP 192.168.1.1

When you're finished testing, make sure to remove the hack.

Files

Finally, here are samples of all of the files modified in this tutorial. See Part I for the other files.

  • iptables.uprules: The default rules to apply to the firewall
  • configfirewall.sh: The firewall configuration executed when a user connects or disconnects from OpenVPN
How to configure a local firewall for OpenVPN

The following tip was developed using Ubuntu 9.1x (Hardy Heron) with OpenVPn 2.1rc19.


Once you're done with this tutorial, make sure to read Part II, which includes some crucial updates.

There are dozens of guides around that describe how to optimally configure the iptables firewall on Linux for OpenVPN. There's even a script installed by default that is extremely well-commented and shows to how close down the firewall, then open up only very selected ports and protocols for optimal browsing. However, all of those guides assume that the machine on which OpenVPN is installed is also the firewall separating an external network (the DMZ) from an internal one. Well, what if you have a dedicated firewall and run the OpenVPN server on a machine running in the internal network?

This tutorial assumes that you've already followed the instructions for setting up OpenVPN and that you've also set up a Public Key Infrastructure (PKI). That means that access to your internal network via OpenVPN is secured and will only authorize users that have a proper certificate and password.

All of the files and scripts mentioned in this tutorial are available for download as files at the end of the article.

Access for all!

Since the external firewall routes requests to OpenVPN directly to the internal machine, it cannot be used to restrict the actions of users that are tunneling into the internal network. Luckily, the default behavior is that users only have access to the OpenVPN server itself, which gives you time to consider how, exactly, you want to open things up.

Here are some questions you need to answer:

  1. To which machines should users have access?
  2. On which ports and protocols should users have access?
  3. Which users should have which access?

For many organizations, the whole point of using OpenVPN is to let users work as if they are on the internal network, but from outside the physical office. In that case, the answers to the questions above will in many cases be:

  1. All of them
  2. All of them
  3. All users should have all access

Let's take care of that trivial case first, then. Execute sudo iptables -nL to show the current firewall configuration. You should see something like the following:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

This table indicates that all input, output and forward requests are accepted. OUTPUT requests are not interesting for this exercise, as they are generated by software running on the server itself, but INPUT and FORWARD requests bear more scrutiny. It looks like the firewall is already configured to allow access to everything your users need: There are no restrictions on inputs, which means that the firewall will allow requests on all ports and protocols for the local machine. There are likewise no restrictions on forwards, which means that requests to other IP addresses in the same subnet will be forwarded to those machines.

So, if FORWARDS are being, well, forwarded, why can't you ping any other machines in the same subnet? Once you know the answer, it's obvious: It's because the firewall isn't the one blocking forward requests. It's because IP forwarding is a networking feature that must be explicitly enabled in the networking configuration. The article How to enable IP Forwarding will help you get this option configured, but the crux of the change is shown below.

Since you'll probably want to make this change permanent, execute sudo vi /etc/sysctl.conf and remove the comment from the front of the line containing net.ipv4.ip_forward = 1. Restart networking by executing sudo /etc/init.d/networking restart and you'll be good to go.

VIP Members Only.

The default network is now set up for smaller installations where everybody has the same permissions everywhere. What if, however, your needs are a little more complex? What if you have some users on your VPN that should only have access to certain resources i.e. certain ports and protocols?

In that case, you'll have to use a different approach: Perhaps something like the following:

  1. Close the firewall by default, including access to the OpenVPN machine itself. That means that authorized users will be able to establish a tunnel using OpenVPN, but that they won't be able to do anything else until the firewall is opened again in the ensuing steps.
  2. Determine which user has connected/authorized through OpenVPN.
  3. Determine the set of IP addresses/ports/protocols to which that user should be given access.
  4. Open the firewall for only those IP addresses/ports/protocols and only from the client IP address for that user's current tunneling session.
  5. When the user disconnects, close the firewall for that client IP address.

The first step is to close the firewall by default. As you can see from the iptables listing above, the firewall accepts all INPUT connections by default. You're probably not an expert on iptables configuration (or you wouldn't be here). There are two ways to get the settings you need:

  1. Execute shell commands to iptables to set up the firewall
  2. Import an iptables configuration from a dump file

There's really not much difference, but this tutorial opted for the second option. Once you've got a default firewall set up to your liking, use iptables-save to dump out the rules to a file named /etc/iptables.uprules (naturally, you can use whatever file name you like; it just has to match the reference from the script below). If this is all very confusing, the values below set up a closed firewall for you, which is probably what you want.

*filter
:INPUT DROP [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i eth0 -j ACCEPT
-A INPUT -i lo -j ACCEPT
COMMIT

Though FORWARD and OUTPUT are still accepted unconditionally, all requests to INPUT are dropped. The two rules for eth0 and lo make sure that the machine can communicate with itself. Now that you've got the rules you need, you want to somehow alter the default configuration of the firewall.

If you guessed that the next step is to edit /etc/iptables/default.conf or /etc/default/iptables.conf, you'd be wrong. That's pretty intuitive, but wrong. On the latest versions of Ubuntu, networking setup like firewall configuration is best accomplished by adding a script that is executed just before the networking interface is established. This guarantees that the default firewall rules are in place before the network is in any way accessible. To do this, add a file called iptables.sh to the /etc/network/if-pre-up.d/ folder; Add the following lines to it:

#!/bin/sh

iptables-restore < /etc/iptables.uprules

exit 0

This is a super-simple script that loads the firewall configuration from the file you just created above. The iptables-restore command is convenient because it replaces the whole configuration, so you don't have to do any resetting of your own.

Save the file and execute sudo chmod +x /etc/netwokr/if-pre-up.d/iptables.sh to make it executable. Restart networking by executing sudo /etc/init.d/networking restart.

A call to sudo iptables -nL should now elicit the following output (the main changes are highlighted):

Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Congratulations! You've succeeded in locking out everybody again, but in a different way.

Separating the Wheat from the Chaff

How do you get back that coveted VIP status that you had just seconds ago? Now you're up to step (2) above: "Determine the user connected through OpenVPN". The basic strategy here is to key on the unique name in the SSL certificate authorized by OpenVPN. For each different group of permissions (IP addresses/ports/protocols) that you want to grant, create a file with the names of people who belong to that group, one name per line. For example:

Joe_Jackson
Phil_Hartman
Jill_Meikenson
Horst_Buchholz
Susan_B_Lazy

This is just one very simple solution to the problem of determining membership. Some installations with much larger user bases might want to instead bind to an external lookup using LDAP or an already existing MySQL database or something similar. That's obviously beyond the scope of this tutorial, though.

You're now going to need a script that will use these lists to determine which firewall rules to execute. I've added the general form of that script below, with matching for "employees" and "strangers" and TODO statements indicating where you need to extend the script for your own purposes:

function inlist
{
  less `dirname $0`/$1 | egrep "^${CLIENTCERT}$" > /dev/null
  if [ $? -eq 0 ]; then
    return 0
  else
    return 1
  fi
}

function get_next_matching_firewall_rule
{
  ip_address=$1

  RULE="`iptables -L INPUT -n --line-numbers | grep $ip_address | head -n 1`"
}

function drop_rule_from_iptables
{
  rule="$1"
  echo "  Drop rule [$rule]"
  line_number=`echo "$rule" | awk '{print $1}'`
  iptables -D INPUT $line_number
}

function add_port_to_iptables
{
  source_ip=$1
  destination_ip=$2
  protocol=$3
  port=$4

  iptables -A INPUT -i tun0 -s $source_ip -d $destination_ip -p $protocol --dport $port -j ACCEPT
}

function add_destination_to_iptables
{
  source_ip=$1
  destination_ip=$2
  iptables -A INPUT -i tun0 -s $source_ip -d $destination_ip -j ACCEPT
}

function open_firewall_for_strangers
{
  echo "  Add route for DNS"
  add_port_to_iptables $CLIENTIP 192.168.1.1 "UDP" 53

  echo "  Add route for Windows shares"
  add_port_to_iptables $CLIENTIP 192.168.1.5 "TCP" 139
  add_port_to_iptables $CLIENTIP 192.168.1.5 "TCP" 445

  return 0
}

function open_firewall_for_employees
{
  echo "  Add routes for all ip addresses"
  iptables -A INPUT -i tun0 -s $CLIENTIP -j ACCEPT
  return 0
}

function open_firewall
{
  echo "Opening firewall for $CLIENTCERT @ [$CLIENTIP]"
# TODO Add filtering for other lists, if desired
# inlist "MYGROUP.list"
#if [ $? -eq 0 ]; then
#  echo "  Certificate found in MYGROUP list"
#  open_firewall_for_MYGROUP
#  return 0
#else
  inlist "strangers.list"
  if [ $? -eq 0 ]; then
    echo "  Certificate found in strangers list"
    open_firewall_for_strangers
    return 0
  else
    inlist "employees.list"
    if [ $? -eq 0 ]; then
      echo "  Certificate found in employee list"
      open_firewall_for_employees
      return 0
    else
      echo "  Certificate not found in any list"
      return 1
    fi
  fi
}

function close_firewall
{
  echo "Closing firewall for [$CLIENTIP]"
  get_next_matching_firewall_rule $CLIENTIP

  while [ -n "$RULE" ]
  do
    drop_rule_from_iptables "$RULE"
    get_next_matching_firewall_rule $ip_address
  done
}

# Main

OPERATION=$1
CLIENTIP=$2
CLIENTCERT=$3

case "$1" in
  add)
    close_firewall
    open_firewall
    ;;

  update)
    close_firewall
    open_firewall
    ;;

  delete)
    close_firewall
    ;;
  *)
    echo "Unknown operation"
    exit 1
esac

exit $?

Some explanation for those who haven't scripted in bash much before:

  • The whole script executes from the case statement at the end of the script. Note that in all recognized cases, the firewall is first closed just to make sure that there are no lingering entries for the given client's IP address.
  • A call to close_firewall simply removes all rules for the given client's IP address, in which case the default DROP action on INPUTS will block all incoming traffic from the address.
  • A call to open_firewall tries to find the user in one of the files. If successful, the rules for that file are applied to the firewall.
  • In this case, if the user is a "stranger", they only have access the DNS server in the internal network (for name lookups) and to the Windows shares on one other machine.
  • If the user is an "employee", then the script adds a rule to allow all ports and all protocols to all IP addresses for that person and restores full access rights.
  • Again, there are a dozen ways of determining membership and it's doubtful that bash is the best language in which to program more complex membership tests. The best language to use is the one you know and the one that runs on your server. ;-)

Finally, you need to tell OpenVPN to run your script whenever it has authorized a connection. Execute sudo vi /etc/openvpn/server.conf and add or modify the following line:

learn-address /etc/openvpn/configfirewall.sh

Restart OpenVPN with sudo /etc/init.d/openvpn retart and you're done! Your OpenVPN server now not only authorizes users but also locks down the firewall to allow only those services for which a user has permission.

Files

Finally, here are samples of all of the files used in this tutorial.

  • iptables.sh: The script to execute when just before the network starts
  • iptables.uprules: The default rules to apply to the firewall
  • configfirewall.sh: The firewall configuration executed when a user connects or disconnects from OpenVPN
  • employees.list: A list of employees against which to match

Once you're done with this tutorial, make sure to read Part II, which includes some crucial updates.

Create indexes for all foreign keys in SQL Server

At Encodo, we use Quino to model our databases for our projects when we can. Quino spoils us by automatically taking care of a lot of the grunt work involved in creating and maintaining a schema that performs well. Automatically creating indexes for foreign key fields is such a nicety.

On a recent project, we decided to try using Microsoft's Entity Framework (EF) instead of embedding Quino. Unlike Quino, EF uses the database as the model, so we had to get used to writing models using the Microsoft SQL Server Management Studio instead of writing them in C#. We quickly discovered that we weren't so good at remembering to dot our i's and cross our t's in the database. For example, once we migrated the original database to the new format (both approximately 6GB), we ran into timeout issues on a lot of queries and were mystified for a few minutes.

That's when it dawned on us that we had no indexes for any of our foreign keys because we'd gotten so accustomed to the modeling tools creating them for us. Creating a constraint using the diagram editor in the SQL Server Studio does nothing of the sort nor does it offer a quick way of saying "create an index for this constraint"; instead, you have to create each index manually. Not liking the sound of that, we came up with the following script instead; it builds an SQL-script that creates missing foreign-key indexes. To use it, execute the following script in a given database to generate database-specific commands, then copy the output and execute that to actually create the missing indexes.

--  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- 
-- Executing this script will generate a create-script for inserting indexes
-- over all tables in the database.
 --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- 
Select
'IF NOT EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N''[dbo].['
+ tab.[name]
+ ']'') AND name = N''IX_'
+ cols.[name]
+ ''') '
+ 'CREATE NONCLUSTERED INDEX [IX_'
+ cols.[name]
+ '] ON [dbo].['
+ tab.[name]
+ ']( ['
+ cols.[name]
+ '] ASC ) WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]'
From sys.foreign_keys keys
Inner Join sys.foreign_key_columns keyCols
 On keys.object_id = keyCols.constraint_object_id
Inner Join sys.columns cols
 On keyCols.parent_object_id = cols.object_id
 And keyCols.parent_column_id = cols.column_id
Inner Join sys.tables tab
 On keyCols.parent_object_id = tab.object_id
Order by tab.[name], cols.[name]
Eject/Change a CD from Windows inside a XEN-VM using VNC

At Encodo, we currently run Debian Etch on our servers, with a Xen hypervisor managing a bunch of individual virtual machines (VMs). Most of the VMs also run Debian Etch, but one of them runs Windows Server 2003 instead. We use this machine for testing integration with Microsoft technologies like Sharepoint, Exchange and so on. Recently, we had to re-install the Exchange instance on that server and were faced with the problem of having to change the CD without rebooting the VM. Luckily, we found the article, Xen 3.0.3 change cdrom with windows 2003, which cryptically describes how to do this. The instructions describe pressing ctrl+alt+1, but where?

The trick is to realize that they are assuming three things:

  1. You have configured the VM to provide a VNC port
  2. You have attached to said port with a VNC viewer
  3. You realize that you can't press these keys directly, but must select them from the viewer's system menu (as shown below).

Before you do anything, verify that you have made the physical CD/DVD available to the machine, by specifying something like the following in the XEN configuration file for the VM:

disk = [ 'file:/home/xen/domains/burken/disk1.img,ioemu:hda,w', 'phy:/dev/cdrom,hdc:cdrom,r' ]

The first disk (disk1.img) is a disk image for the system itself; the second disk (hdc:cdrom) is the physical CD/DVD. Until you see the CD inside the VM, you don't have to even worry about trying to eject it.

You also need to make sure the VNC port is available, again with a line in the configuration:

vnc=1

If you make any changes to the configuration, you'll need to restart the VM before you see the effects. Use the additional configuration option called vncpasswd to lock down the VNC port.

Once you can see the CD within the VM and you can open a connection with the VNC viewer, you're ready to actually follow the instructions in the post linked above:

  1. type ctrl-alt-2
  2. type help
  3. type eject hdb
  4. Change the CD manually (at the server itself)
  5. type change hdb /dev/cdrom1

At this point, you might think you're done, but the first step is a stumbling block as you don't actually type ctrl and alt; instead, you select them from the system menu, as illustrated below:

[image](https://data.encodo.ch/news/attachments/entry/162/2009-03-20_1024_vnc_switch_to_console.swf)

That's it; you should see the new CD in the VM and you can continue with your installation.



  1. Where /dev/cdrom corresponds to the CD/DVD drive in question.

Tech Tips: Stopping "Spam"

Email used to be a very efficient means of communication. Nowadays, it's become much less efficient due to spam. People who used to have a few messages per day -- from people they knew and about topics in which they were interested -- now find themselves inundated with hundreds of messages per day, all trying to sell them something. This extra mail is called "spam"; here's an official definition:

spam: Unsolicited e-mail, often of a commercial nature, sent indiscriminately to multiple mailing lists, individuals, or newsgroups; junk e-mail.

Why am I getting Spam?

The first thing you have to realize is that, if you're going to use email, you're going to get spam. Even if you jealously guard your email address (as you should!), it will still leak out eventually. You may be thinking that you're safe because you only give out your email address to people and web-sites that you trust. You're not foolish enough to just give your address to spammers, so you should only get legitimate mail.

Unfortunately, it's very easy for your email address to become public. In order to participate on most web sites, you must provide a valid email address. The web site sends an email to the address you gave them and enables your account only when you confirm that you received it. Now you're stuck with the choice between giving these web sites your email address and not participating at all. Sometimes -- as with online stores -- you don't even really have a choice. In these cases, you'll have to determine whether the web site is trustworthy or not.

Many sites have privacy policies, in which they explain how, when and where they will use your personal information, including your email address. If you feel you can trust the site, then go ahead and give them your email address. If you don't like the conditions, you should not do business with that web site or use one of the measures outlined below to protect your real address from spam.

Less-reputable companies and stores will sell your personal information to spammers. Even if the company doesn't sell your address, it may be stolen from their server. If your address is published on a publicly available web page, then automated software can find it and store it in a spammer's address book.

The harsh reality of using email today is that, as soon as an email address is publicly available, it has been compromised.

It's not Hopeless, is it?

So it sounds pretty hopeless, right? It's not, though; it's just that using email spam-free requires a bit of work and planning and, above all, thinking about what you're doing online. Using the simple rules below, you can still exert some control over your inbox so that you can:

  1. Minimize the amount of spam that gets sent to you.
  2. Minimize how much of that spam actually arrives at your inbox.

Taking Control of Your Inbox

Before you start to clean up your email, you need to take a long, hard look at your inbox. Is it full of spam? Does it take you longer to sort through spam than it does to read your actual email? Do you sometimes throw things away that you should have read?

Well, the very first thing you should do is turn on spam filtering in your client. Almost all Windows and OS X applications as well many web sites offer some form of this feature. Spam filtering means that the program you use to retreive mail checks each mail as it comes in and puts it in a "spam" or "junk" folder if it thinks it's spam. You can train the filter over time in order to improve its accuracy.

If you're already filtering spam and your inbox is still full, you may need to bite the bullet and get a new email address, closing the "spammy" email account for good. This is a drastic measure, to be sure, but no amount of organization is going to help you if your inbox is already severely compromised. Let's assume, however, that you either have a new account or that your existing account is still okay.

So, how do you keep it that way?

The first step is to make a list of addresses to which you send and from which you would like to receive mail. Most people have at most two or three main groups of addresses:

  1. Private: Includes family & friends; can be trusted not to send spam.
  2. Business: Includes clients who will most likely not send you spam.
  3. Online: Includes everything else, like online forums, eBay, Amazon, Google and so on. This group includes possible spammers.

Now here's the key: email addresses are not like home addresses: it doesn't cost you anything to have more than one of them. The best solution is to choose a separate email address for each of the three groups above.

You probably already have a business email address; use this only for business. Stop using it for anything else.

If you don't already have a private email address, you can easily get a free one at Yahoo, Google or Hotmail. The advantage of getting an account with one of these providers is that they do a very good job of filtering spam because, as mentioned above, you will still get spam, no matter what you do.

You may already have a private address from the ISP (Internet Service Provider) that provides your internet connection at home (e.g. Cablecom). You can use this address but be aware that, should you change your ISP, you will have to change your email address as well.

Finally, you should choose another email address for your online needs and use only that one when you have to register on a web site.

Advanced Spam Fighting with Spam Gourmet

The solution for handling "online" usage of your email works fine. However, it does suffer from a single point of failure: if one site to which you subscribed misbehaves, you've suddenly got an inbox full of spam again -- even if the other 99% of the sites you work with are okay. It would actually be nice to somehow have a separate email address for each site so that you can just stop mail from that single, misbehaving site. But that would be a tremendous amount of work, wouldn't it?

In steps SpamGourmet (available in multiple languages). This site provides exactly that service for you: you open an account there and give it your "real" email address (which would be your online address above). They provide you with an unlimited number of addresses that you can use with your account. For example, let's assume you opened an account for the user "myself". Your base address is then myself@spamgourmet.com. Now, suppose you want to purchase something at "Joe's Shady Shack O' Hardware". They've got great prices, but you don't really trust them not to spam you. So you can just make an address for them called joes_shack.10.myself@spamgourmet.com. SpamGourmet will detect that the mail should go to the account for "myself" and forward it to your real email address. So far, so good.

Let's take a closer look at the special address we gave to Joe and his Shady Shack O' Hardware. You can see that the email address ends in myself@spamgourmet.com; that always has to be there so that SpamGourmet knows to which account the mail is being sent. In front of that are two other pieces of information: joes_shack, which is a unique identifier for that site and 10, which indicates to SpamGourmet how many mails Joe is allowed to send to you. Since you only expect Joe to send you one or two mails, you don't have to choose too high a number. However, if Joe starts spamming you, you'll get at most 10 mails from him before SpamGourmet stops forwarding mail from him. It's that simple!

The unique identifier can be anything that helps you remember at which site you used the address. Here are a few more examples:

msdn.5.myself@spamgourmet.com

Microsoft's MSDN web site can send at most 5 mails.

cablecom.20.myself@spamgourmet.com

Cablecom can send at most 20 mails.

There are more advanced features on SpamGourmet as well:

  • You can change the number of mails an address can send.
  • You can set which senders are allowed to use an address.
  • You can set an address as "trusted" so that mails are always forwarded from there.

With the information in this article, you should be much better equipped to fight spam. You'll still get some spam, but hopefully you'll get much less!


There is much, much more to tell about spam, from "gray listing", which explains why some mail takes so long to arrive, to installing spam filtering software like D-Spam on your server if you have a private mail account. A discussion of those topics are quite beyond a typical "tech tip", but would fit into our blogs quite well. Let us know if you're interested in hearing more.

Tech Tipp: "Spam" stoppen

Es war einmal - eine neue effiziente Kommunikationstechnologie mit Namen E-Mail. Heute hat uns Spam die Effizienz weitgehend zunichte gemacht. Leute, die früher einige E-Mail am Tag bekamen - von Freunden und Geschäftspartnern über Themen an denen sie interessiert waren - können heute mit hunderten von Mitteilungen überschwemmt werden und jeder will etwas verkaufen. Diese zusätzlichen ungewollten E-Mails nennt man 'Spam', hier eine offizielle Definition:

spam: Unsolicited e-mail, often of a commercial nature, sent indiscriminately to multiple mailing lists, individuals, or newsgroups; junk e-mail.

und auf Deutsch: Nicht angeforderte E-Mail Mitteilungen, oftmals komerzieller Natur, wahllos an verschiedene Adresslisten, Einzeladressen oder Newsgruppen versandt.

Wieso bekomme ich plötzlich Spam?

Grundsätzlich kann man davon ausgehen, dass jeder der sich dafür entscheidet E-Mail zu nutzen auch Spam erhalten wird. Da nützt keine Geheimniskrämerei und exklusive Kontrolle der eigenen Adresse, irgendwann wird das Geheimnis gelüftet. Zu denken man sei sicher, weil man die Adresse nur an ausgewählte Leute und vertrauenswürdige Websites herausgibt ist ein Trugschluss.

Leider ist es für eine E-Mail Adresse sehr einfach publik zu werden. Newsletter abonnieren, Foren abfragen, Services nutzen - alle wollen eine E-Mail Adresse von Ihnen. Meist schicken diese Services dann eine Mitteilung an die angegebene Adresse und der Service funktioniert nur, wenn der Empfang der Meldung quittiert worden ist. Somit kann man sich auch nicht mit einer falschen Adresse aus dem Dilemma retten. Will man den Service nutzen, bleibt einem nichts anderes übrig als eine funktionierende korrekte E-Mail Adresse anzugeben.

Viele Websites bieten Informationen an, in welcher Form die angegebenen Daten (inkl. E-Mail Adresse) genutzt werden. Ist man damit einverstanden, ist alles klar und einer problemlosen Kommunikation steht nichts im Wege. Gefallen Ihnen die Bedingungen nicht, dann verzichten Sie auf den Service oder ergreifen eine der weiter unten beschriebenen Massnahmen um ihre wertvolle E-Mail Adresse vor Spam zu schützen.

Wenig vertrauenswürdige Anbieter verkaufen Ihre persönlichen Daten an Dritte, die sie für Spam missbrauchen. Daten werden auch von Servern oder anderen Datenträgern gestohlen. Das sorgt immer mal wieder für Schlagzeilen - so letzthin in den Medien präsent Kundendaten einer Bank. Unauffälliger geschieht es, wenn automatisierte Software das Internet auf öffentlich registrierte Adressen absucht und diese dann abspeichert.

Die harte Tatsache heute ist, dass eine Adresse, sobald öffentlich zugänglich, auch schon gefährdet ist.

Aber es gibt doch Hoffnung, oder?

Ganz so düster ist die Lage nicht - mit etwas Planung, Einsatz und Voraussicht kann man E-Mail auch heute noch weitgehend Spam-frei nutzen. Hält man sich an die untenstehenden einfachen Regeln haben Sie Ihre Mailbox unter Kontrolle und

  1. reduzieren die Menge Spam, die sie erhalten
  2. reduzieren die Menge Spam, die es wirklich bis in Ihre Mailbox schafft.

Die eigene Mailbox unter Kontrolle haben

Bevor Sie anfangen aufzuräumen schauen Sie sich Ihre Inbox einmal ganz genau an. Wieviel Spam ist da drin? Brauchen Sie mehr Zeit um Spam auszusortieren als die echten E-Mails zu lesen? Passiert es hin und wieder, dass Sie Mitteilungen gelöscht haben, die sie hätten lesen sollen? Können Sie diese Fragen mit Ja beantworten besteht Handlungsbedarf.

Zuallererst stellen wir sicher, dass der im Mailprogramm eingebaute Spamfilter eingeschaltet ist. Fast alle E-Mail Programme für Windows und OS X, sowie viele Websites bieten diesen Service an. Ein Spamfilter empfängt jede hereinkommende Mitteilung und sortiert diejenigen Mitteilungen aus, die wahrscheinlich Spam sind. Diese Mitteilungen werden in einen separaten Spam- oder Junk-Ordner verschoben. Diese Filter lassen sich trainieren, so dass sie im Laufe der Zeit immer besser werden.

Falls sie das schon lange machen und Ihre Inbox trotzdem voller Spam ist, müssen Sie wohl in den sauren Apfel beissen und sich eine neue E-Mail Adresse beschaffen. Die Spam-verseuchte Adresse legen sie am Besten still. Ja, das ist eine drastische Massnahme und unpopulär dazu. Aber eine einmal völlig verspamte E-Mail Adresse kann man nicht wieder blankputzen. Gehen wir also davon aus, dass Sie eine neue oder noch gute E-Mail Adresse haben.

Wie stellt man also sicher, dass das so bleibt?

Erstellen Sie eine Liste der Adressen mit denen sie korrespondieren und das auch weiterhin tun wollen. Die meisten Leute haben zwei bis drei Gruppen von Empfängern:

  1. Privates: Familie und Freunde; vertrauenswürdige Leute, die mir kein Spam senden.
  2. Geschäftliches: Kunden und Lieferanten, die mir auch kein Spam senden werden.
  3. Online: Alles andere, wie Online Foren, Auktionsplattformen, Amazon, Google und so weiter. In dieser Gruppe gibt es potenzielle Spammer.

Und jetzt der Schlüssel zum Ganzen: Da eine E-Mail Adresse nicht wie eine Postadresse ist, hindert Sie nichts daran, sich mehrere davon zu beschaffen. Die Beste Möglichkeit Spam zu kontrollieren ist, für jede der Adressatengruppen separate E-Mail Adressen zu verwenden.

Für die geschäftlichen Kontakte dürfte das höchstwahrscheinlich bereits der Fall sein. Benutzen Sie diese Geschäftsadresse aber für nichts anderes!

Haben Sie noch keine private E-Mail Adresse kann man eine solche gratis bei verschiedenen Anbietern lösen, z.B. Yahoo, Google or Hotmail.

Hier kommen Sie auch in den Genuss der bereits wirklich gut trainierten Spamfilter die eingesetzt werden. Denn Spam wird immer noch an Sie gesandt, das lässt sich nicht vermeiden - er schafft es aber dank dem Filter nicht mehr bis in den Posteingang.

Sehr wahrscheinlich haben sie auch eine private Adresse von Ihrem Internet Service Anbieter erhalten, wo Sie ihr Abonnement für den Internetzugang haben (z.B. Cablecom). Diese Adresse kann man auch benutzen. Seien Sie sich einfach bewusst, dass diese Adresse sich ändern würde, wenn Sie den Anbieter wechseln.

Zu guter Letzt brauchen Sie eine weitere E-Mail Adresse für alle Online Bedürfnisse. Benutzen Sie nur diese Adresse, wenn Sie sich online irgendwo anmelden oder registrieren.

Spam Bekämpfung für Fortgeschrittene mit Spam Gourmet

Eigentlich funktioniert die Lösung mit der separaten Adresse für Online Belange gut - ausser eine Seite, auf der Sie registriert sind versagt und sofort ist der Posteingang wieder mit Spam gefüllt - auch wenn die anderen 99% der Seiten, die Sie nutzen sich vorbildlich verhalten. Es wäre also wünschenswert, dass man irgendwie eine separate Adresse für jede Website hätte. Dann könnte man bei einem solchen Versagen die betroffene Adresse stilllegen. Mit Recht denken Sie jetzt, dass das riesiger Aufwand ist.

Und hier kommt SpamGourmet (verfügbar in verschiedenen Sprachen). Diese Webseite liefert genau den gesuchten Service: Man eröffnet einen Benutzeraccount mit der 'echten' E-Mailadresse (das wäre in dem Fall ihre Adresse für Online Registrationen), SpamGourmet stellt dann eine unbeschränkte Anzahl Adressen zur Verfügung. Ein Beispiel: Eröffnen wir ein Konto für den Benutzer "myself". Die Basisadresse ist dann myself@spamgourmet.com. Wir wollen nun etwas bei "Fredi's Super Sport Discount" bestellen. Die Preise sind sensationell, aber wir sind nicht überzeugt, dass die uns nicht mit Spam eindecken würden. Also machen wir uns jetzt eine Adresse fredi-super-sport.10.myself@spamgourmet.com. Spamgourmet weiss nun, wenn eine Meldung mit dieser Adresse eingeht, dass wir hinter dem Benutzer "myself" stecken und schickt uns die Mitteilung an unsere 'echte' Adresse weiter. So weit so gut.

Schauen wir uns die Adresse, die wir bei Fredi im Super Discount angegeben haben genauer an. Die Adresse endet mit myself@spamgourmet.com; diese Information benötigt SpamGourmet um die Mitteilung dem richtigen Benutzer zu senden. Davor stehen zwei weitere Informationen: fredi-super-sport, als Identifikation für die Seite wo wir die Adresse benutzt haben und 10, als Information für SpamGourmet wieviele E-Mails wir von Fredi erhalten wollen. Wir rechnen damit, dass es nicht mehr als eine oder zwei E-Mails braucht um unsere Bestellung abzuwickeln, also müssen wir keine sehr hohe Nummer für die SpamGourmet Adresse angeben. Sollte Fredi aber wirklich anfangen uns mit Spam einzudecken ist spätestens nach 10 E-Mails Schluss! SpamGourmet leitet die Spam-Mitteilungen nicht weiter. So einfach ist das.

Der Adressteil mit der Identifikation soll helfen zu erkennen, wo die Adresse angegeben wurde. Hier noch einige Beispiele:

msdn.5.myself@spamgourmet.com

die Microsoft MSDN Website kann maximal 5 E-Mails senden.

cablecom.20.myself@spamgourmet.com

Cablecom kann maximal 20 E-Mails senden.

SpamGourmet kann aber noch mehr:

  • Ich kann die Anzahl der Mitteilungen, die eine Adresse senden kann ändern.
  • Ich kann bestimmen, welche Absender eine Adresse benutzen können.
  • Ich kann eine Adresse als 'vertrauenswürdig' markieren, so dass alle Mitteilungen dieses Absenders an mich weitergeleitet werden.

Mit diesen Informationen hoffen wir, dass Sie besser gerüstet sind im Kampf gegen Spam. Etwas werden Sie wohl immer erhalten - aber hoffentlich drastisch weniger!


Natürlich gäbe es zu diesem leidigen Thema noch viel mehr zu sagen, von "gray listing", das erklärt warum es manchmal Ewigkeiten dauert bis E-Mails ankommen, bis zur Installation von Spamfilter Software, wie D-Spam auf dem eigenen Server. Dies sprengt aber den Rahmen eines Tech Tipps bei weitem - könnte aber durchaus in unserem Blog einen Platz erhalten. Interessiert? Melden Sie sich bei uns und wir helfen Ihnen noch einen Schritt weiter im Kampf gegen Spam.

Tech Tipp: Newsletter vs. Newsfeed

Viele von uns werden täglich von der Informationsflut aus dem Internet überrollt. Dieser Artikel soll zwei Technologien zeigen, wie man diese Flut eindämmen oder mindestens den Kopf wieder über Wasser bekommen kann.

Will man über ein Thema, eine Website oder eine Firma auf dem Laufenden bleiben, müsste man regelmässig die entsprechenden Websites besuchen und nach neuen, interessanten Informationen absuchen. Dies nimmt schnell viel Zeit in Anspruch. Was tun, wenn man auf dem Laufenden bleiben will ohne stundenlange Recherchen?

Viele Websites stellen Funktionen zur Verfügung, um gezielt Informationen abzurufen, ohne die Website regelmässig mit dem Browser zu besuchen. Dies ermöglicht es, sich auf die Informationen (News) zu fokussieren, die wirklich interessieren.

Hier stellen wir zwei gängige Techniken vor um Informationen gezielt zu beziehen:

Newsletter (Email-Abonnemente)

Viele Firmenwebsites bieten Email-Abonnemente (eng. Email-Subscriptions) an. Nachdem die eigene Emailadresse in einer Anmeldung eingetragen wurde, werden mehr oder weniger regelmässig die News zur jeweiligen Webseite bzw. Firma via Email zugestellt und können ganz normal im Posteingang gelesen werden. Das dürfte Ihnen bekannt vorkommen - genau so haben Sie sich auch für den Encodo Newsletter eingetragen. Diese Technik gehört zu den sog. Push-Technologien, da Sie selbst - nach dem erstmaligen Eintragen Ihrer Emailadresse keine weiteren Aktionen mehr zu tätigen brauchen, um die Informationen zu bekommen; die Informationen werden Ihnen sobald verfügbar geliefert.

Damit eine Website Email-Abonnemente umsetzen kann, muss die Firma alle Emailadressen der Abonennten kennen und bei sich speichern um die News zuzustellen. Ob diese Emailadressen dann auch noch anderweitig verwendet (z.B. für Werbung, Spam) oder sogar weitergegeben werden kann der Betreiber der Website selbst entscheiden. Es ist daher empfehlenswert, das man seine Emailadresse nur auf Websites einträgt, welchen man auch einigermassen vertraut, dass mit diesen persönlichen Daten kein Unfug getrieben wird. Falls bei der Anmeldung für den E-Mail Service die Möglichkeit besteht auszuwählen ob man Werbung erhalten möchte oder ähnliches, dann nutzen Sie doch diese Chance und überlegen sich gut ob Sie das möchten. Bei Encodo können Sie den Newsletter hier abonnieren...

Newsfeeds

Im Gegensatz zum Newsletter handelt es sich bei den Newsfeeds um eine Pull-Technologie. Das bedeutet, dass Sie regelmässig aktiv werden müssen, um Informationen abzufragen. Bei Newsfeeds - auch RSS-Newsfeeds oder RSS-Feeds genannt 1 handelt es sich um ein Standardformat zur Auflistung von Newsbeiträgen. Hier stellt die Website spezielle URL-Adressen zur Verfügung, über welche der Newsfeed abgefragt werden kann.

Der Leser kann diese URLs in eine spezielle "Newsreader" Software eintragen. Ist das einmal gemacht, zeigt das Newsreader-Programm automatisch an, sobald es bei einem der Newsfeeds neue Beiträge gibt. Diese können dann direkt im Newsreader-Programm gelesen werden.

Besonders komfortabel ist es, wenn das eigene Email-Programm Newsfeeds unterstützt. In diesem Fall wird dann das Email-Programm auch zum Newsreader und man kann Emails und Newsfeeds komfortabel am gleichen Ort lesen.

Da Newsfeeds vom Leser abgefragt werden und nicht automatisch zugestellt werden, muss keine Emailadresse hinterlegt werden. Somit ist auch sichergestellt, dass damit kein Unfug getrieben werden kann.

imageNewsfeeds werden meist mit einem speziellen Symbol einem orangen Quadrat mit Radiowellen signalisiert (siehe rechts). Daneben gibt es auch noch die Bezeichnungen "XML", "RSS" oder "ATOM" welche häufig (aber nicht immer) auf orangem Grund geschrieben werden. Heutzutage werden bei weitem nicht nur klassische News als Newsfeeds angeboten. Von 'neue Forumsbeiträge' bis hin zu Radio- oder Fernsehsendungen 2 (=Podcasts) ist alles als Newsfeed verfügbar. Sogar Websites können Newsfeeds anderer Websites in den eigenen Inhalt integrieren.

Und wie kommt man jetzt an die Newsfeeds? So wird's gemacht:

  • Auf der Webseite mit der rechten Maustaste auf das orange Newsfeed-Symbol klicken und "Adresse kopieren" wählen.
  • Die Software zum Lesen von Newsfeeds (siehe Liste unten) starten.
  • Auf "Neuer Newsfeed abonnieren" klicken (kann je nach Software etwas anders heissen).
  • Ins Adressfeld die kopiert Adresse einfügen und "Ok" klicken.
  • Von nun an kann der Newsfeed in diesem Programm fast wie ein E-mail gelesen werden.

Hier einige Beispiele für Newsreader-Software:

Live-Bookmarks

In obiger Liste fällt auf, das man Newsfeeds auch mit den gängigen Browsern lesen kann. Nun haben wir in der Einleitung gesagt, das wir die Informationen ohne Browser lesen wollen. Warum hier also wieder Browser als Newfeedreader?

Die aktuellen Browserversionen haben eine spezielle Unterstützung für Newsfeeds eingebaut - die sog. "Live-Bookmarks". Trägt man die URL des Newsfeeds im Browser als "Favorit" ein, aktualisiert der Browser die darin enthaltenen Lesezeichen (eng. "Bookmarks") automatisch wenn die "Favoriten" geöffnet werden und man sieht auf einen Blick, bei welchen Newsfeeds es neue Informationen gibt. Gibt es keine neuen Informationen, erspare ich mir den Besuch der Website und bin trotzdem sicher, dass ich nichts verpasst habe.

Newsfeeds in der Sidebar von Windows Vista

In Windows Vista ist es möglich sich Newsfeeds mittels der Windows Sidebar direkt auf dem Desktop anzuzeigen. Leider ist es nicht auf den ersten Blick ersichtlich, wie man hier die Newsfeeds einträgt.

So wirds gemacht:

  • Internet Explorer öffnen.
  • Auf den gewünschten Newsfeed navigieren.
  • "Subscribe this feed" (dt. "Feed abonnieren") klicken um den Newsfeeds in die Feed-Bookmarks des Internet Explorers aufzunehmen.
  • Internet Explorer schliessen.
  • In der Sidebar von Windows Vista das Gadget namens "Feed Headlines" (dt. "Minianwendung Feedschlagzeilen") einblenden.
  • In dessen "Optionen" kann noch eingestellt werden, welche Feeds angezeigt werden.
  • Die Newsfeeds aus dem Internet Explorer werden jetzt direkt im "Feed Headlines" angezeigt und laufend aktualisiert - wie ein Newsticker.

Tipp zum Start mit Newsfeeds

In der Praxis hat es sich bewährt, wenn Newsfeeds im selben Programm wie die Emails gelesen werden können. Um mit Newsfeeeds einzusteigen empfiehlt es sich, zu prüfen ob die aktuelle Version des eigenen Mailprogrammes diese Technologie unterstützt.

Für Leute die sehr viele Newsfeeds verwalten und abfragen, empfiehlt es sich die spezialisierten Newsreader anzuschauen. Diese bieten oftmals noch die eine oder andere Spezialfunktion, welche es in den Emailprogrammen evtl. nicht gibt, wie z.B. "Zeitungsansicht" wo automatisch eine Ansicht ähnlich der Tageszeitungen generiert wird.

Die Live-Bookmarks hingegen haben sich bisher nicht durchgesetzt und fristen eher ein Schattendasein.

Newsfeeds von Encodo

Auch Encodo bietet diverse Newsfeeds an. Zu finden sind Sie auf den entsprechenden Seiten mit dem obigen Symbol markiert.

Ach ja, noch eine Bemerkung für Newsfeeds Anfänger: wenn es die Mailbox mit ungelesenen Newsfeeds füllt nach dem ersten Versuch die neue Möglichkeit zu nutzen, dann waren Sie erfolgreich! Am Anfang erhalten Sie alle verfügbaren Newsfeeds aufs Mal - von da an gibts dann eben sporadisch neue Informationen und die Newsflut ist Vergangenheit.

Entwickeln wir uns doch von 'underinformed and overnewsed' zu 'well-informed and custom-newsfed'.

Weitere Informationen: RSS, Feedreader, Podcasting



  1. RSS = Really Simple Syndication

  2. Beispielsweise bietet das Schweizer Radio und Fernsehen viele der Sendungen als Podcast an, nachdem diese erstmals ausgestrahlt wurden.

Improving XWiki's Search

The XWiki.org site integrates -- and has for a while -- a search that's different from the standard one. It offers users the ability of restricting a search to a specific space instead of searching all spaces at once. Here at Encodo, we've been keeping our weekly reports in the wiki for years and these reports have continuously muddied our search results. The new search is a great improvement over the original.

Everything you see in XWiki is editable in one way or another and that includes the search page/panel. We're going to replace some of these standard pages with our own code.

The Search Form

First we'll create a page for the new search (Textbox etc.). To accomplish that, point your browser to http://yourwiki/xwiki/bin/edit/Main/NewSearch and paste the following lines into the editor:

1 Search

#if(!$request.space)
  #set($space = "All")
#else
  #set($space = $request.space)
#end

#set($spacesText = {})
#set($spaces = $xwiki.spaces)
#set($ok = $spacesText.put("All","All"))
#foreach($space in $spaces)
  #set($ok = $spacesText.put($space,$space))
#end

#macro(spaceoption $space $selectspace $spacesText)
  <option value="$spacesText.get($space)" #if($selectspace == $spacesText.get($space))selected#end>$space</option>
#end

#macro(spaceselect $selectspace $spaces $spacesText)
  <select name="space">
    #spaceoption("All" $selectspace $spacesText)
    #foreach($space in $spaces)
      #spaceoption($space $selectspace $spacesText)
    #end
  </select>
#end

#if($request.getParameter("text"))
  #set($text = $request.getParameter("text"))
#else
  #set($text = "")
#end

#set($utext = $xwiki.getURLEncoded($text))
#if($space == "All")
  #set($url = $xwiki.getURL("Main.WebSearchRss", "view", "xpage=rdf&text=${utext}" ))
#else
  #set($url = $xwiki.getURL("Main.WebSearchRss", "view", "xpage=rdf&space=$space&text=${utext}"))
#end

<div style="float: right;">
  <a href="$url"><img src="$xwiki.getSkinFile("icons/black-rss.png")" border="0px" /></a>
</div>

<form action="">
  {pre}
    <div class="centered">
      Query
      <input type="text" name="text" value="$!text.replace(">", ">").replace("<", "<").replace('"', """)" size="20"/>
      in space #spaceselect($space $spaces $spacesText) <input type="submit" value="Search"/>
    </div>
  {/pre}
</form>

#if($text == "")
  ## No search
#else
  #set($text = $text.replaceAll("'", "''").replaceAll("%", "\\%"))
  #set($datedlist = $xwiki.arrayList)
  
  #set($nbitems = 50)
  
  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  ## Non-admins should not see results from XWiki, Main, Admin 
  ## and Panels spaces. Also exclude the WebPreferences doc.
  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  #if ($xwiki.hasAdminRights())
    #set ($excludedWebs = "")
  #else
    #set ($excludedWebs = "doc.web<>'XWiki' and doc.web<>'Admin' and doc.web<>'Panels' and doc.name<>'WebPreferences' and")
  #end

  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  ## Display only a given space if $request.space is defined
  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  #if($space == "All")
    #set ($webClause = "$excludedWebs")
  #else
    #set ($webClause = "doc.web='$space' and $excludedWebs")
  #end

  #macro(addelement $item $list)
    #if($xwiki.hasAccessLevel("view", $context.user, "${context.database}:${item}"))
      #set($itemdoc = $xwiki.getDocument($item))
      #set($sdate = $xwiki.formatDate($itemdoc.date, "yyyyMMddHHmmss"))
      #set($sitem = "${sdate}${item}")
      #if(!$list.contains($sitem))
        #set($discard = $list.add($sitem))
      #end
    #end
  #end

  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  ## Search in page content
  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  #set ($sql = "where $webClause upper(doc.content) like upper('%$!text%') order by doc.date desc")
  #foreach ($item in $xwiki.searchDocuments($sql , $nbitems, 0))
    #addelement($item $datedlist)
  #end

  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  ## Search in text fields (simple String properties)
  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  #set($sql= ", BaseObject as obj, StringProperty as prop where $webClause obj.name=doc.fullName and prop.id.id = obj.id and upper(prop.value) like upper('%$!text%')")
  #foreach ($item in $xwiki.searchDocuments($sql , $nbitems, 0))
    #addelement($item $datedlist)
  #end

  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  ## Search in big text fields (textarea properties)
  ##  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  -- --
  #set($sql= ", BaseObject as obj, LargeStringProperty as prop where $webClause obj.name=doc.fullName and prop.id.id = obj.id and upper(prop.value) like upper('%$!text%')")
  #foreach ($item in $xwiki.searchDocuments($sql , 50, 0))
    #addelement($item $datedlist)
  #end

  #set($list = $xwiki.arrayList)
  #foreach($item in $xwiki.sort($datedlist))
    #set($ok = $list.add(0, $item.substring(14)))
  #end
  
  #includeInContext("XWiki.Results")
  
#end

The Results Page

The code above gets you a new search form, but it won't work until we update the results page as well. The results of the search page are displayed by the XWiki.Results page. Logging in as Administrator, you can replace the old one with the code-snippet below. Use the edit link on the right side of the search-page (while in edit-mode):

#set($showdata = 0)
#set($formatDate = "yyyy MMMM dd, HH:mm")

## WARNING: Do not add any empty line inside the table element. This will potentially break 
## the Javascript we're using for filtering/sorting columns. It might work in FF but will break
## in other browsers like IE. This is because empty lines add <p class="paragraph"></p> elements
## when rendered.

<table id="searchTableUnique" class="grid sortable filterable doOddEven" cellSpacing=0 cellpadding="0" border="1">
  <tr class="sortHeader">
    <th>Page</th>
    <th width="150" class="selectFilter">Space</th>
    <th width="150">Date</th>
    <th width="150">Last Author</th>
    #if($xwiki.hasAdminRights())
      <th width="210" class="unsortable noFilter">Actions</th>
    #end
  </tr>
  #foreach ($item in $list)
    #set($troubi ="non")
      #if ($xwiki.hasAccessLevel("view", $context.user, "${context.database}:${item}"))
        #set($bentrydoc = $xwiki.getDocument($item))
        #set($cclass = $xwiki.getDocument("XWiki.XWikiComments").getxWikiClass())
        #set($comment = $cclass.newObject())
        #if($xwiki.getWebPreferenceAsInt("commentsorder",1)==0)
          #set($comments = $bentrydoc.getComments())
        #else
          #set($comments = $bentrydoc.getComments(false))
        #end
        #set($createur = $xwiki.getUserName($bentrydoc.author))
        #set($ptitle = $bentrydoc.getDisplayTitle())
        <tr><td align=left>
          #if($comments.size()>0)  
            #set($i = 0)  
            #set($cobj = $comments.get($i))  
            #set($comment = $bentrydoc.display("comment", "view", $cobj))  
            #set($date = $cobj.getXWikiObject().get("date").value)
            #if($date)
              #set($date2 = $!xwiki.formatDate($date,"yyyy MM dd HH:mm:ss")  )
            #end
            #if($bentrydoc)
              #set($date1 = $!xwiki.formatDate($!bentrydoc.date,"yyyy MM dd HH:mm:ss") )
            #end
            #if($date1.equals($date2) )
              [$ptitle>${bentrydoc.web}.$bentrydoc.name] <em>- 1 new comment</em>
              #set($troubi ="oui")
              #set($desc = $cobj.getXWikiObject().get("comment").value)
            #else
              [$bentrydoc.name>${bentrydoc.web}.$bentrydoc.name] #if ($ptitle != $bentrydoc.name) <em>- $ptitle</em>#end
            #end
          #else  
            #set($comment = "")  
            [$bentrydoc.name>${bentrydoc.web}.$bentrydoc.name.replaceAll("@","%40")] #if ($ptitle != $bentrydoc.name) <em>- $ptitle</em>#end
          #end   
        </td><td align=left>
          [$bentrydoc.web>${bentrydoc.web}.WebHome]
        </td><td align=left>
          $xwiki.formatDate($bentrydoc.date,"yyyy MMM dd") at $xwiki.formatDate($bentrydoc.date,"HH:mm")</td><td align=middle>
          #if($troubi =="oui")
            #set($createur = $xwiki.getUserName($cobj.author)   )
          #end
          #if ($createur == "XWikiGuest")
            Guest
          #else
            $createur
          #end
        </td>
        #if($xwiki.hasAdminRights())
          <td>
            <a href="$xwiki.getURL("Admin.CopyDocument", "view", "sourcedoc=${bentrydoc.fullName}")">Copy</a> - <a href="$bentrydoc.getURL("delete")">Delete</a> - <a href="$bentrydoc.getURL("view", "xpage=rename&step=1")">Rename</a> - <a href="$xwiki.getURL("Admin.PageRights", "view","page=${bentrydoc.fullName}")">Rights</a>
          </td>
        #end
      </tr>
    #end
  #end
</table>

If you open http://yourwiki/xwiki/bin/view/Main/NewSearch in your web browser, you should now be able to use the new search. But panel on the right will still be using the old search. That's what we'll fix next.

The Results Page

Log in as administrator (if you aren't already) and go to http://yourwiki/xwiki/bin/view/Panels/. There you can edit/add new Panels (they are all wikipages remember?). Create a new one called "NewSearch", using a "view" panel type.

The content of the panel is:

#panelhiddenheader("Search")
  <form action="/xwiki/bin/view/Main/NewSearch">
    <div id="globalsearch">
      <input id="globalsearchinput" type="text" name="text" value="$msg.get('Search')" size="15" onfocus="if (this.value == '$msg.get('Search')') value=''; this.select();" onblur="if (this.value == '') value='$msg.get('Search')'; this.blur()"/>
 
      <input class="button" value="Search" type="image" src="$xwiki.getSkinFile("go.png")"/>
    </div>
  </form>
#panelfooter()

This is basically a copy of the "Search" panel (the action url differs). The only thing left to do is to register the panel -- or better yet, replace the old search. Open http://yourwiki/xwiki/bin/admin/XWiki/XWikiPreferences (or click on Administration/Preferences) and open the skin section. There you'll find an edit-box to specify the panels to be displayed on the right side. Replace "Panels.Search" with "Panels.NewSearch" and you're done.

From MediaWiki to XWiki part III

As mentioned in the previous article, one of the loose ends to clear up is importing into separate spaces. One of the reasons we moved to XWiki was for the multiple namespace support; it would have been a shame to import all of our existing data into one, gigantic namespace.

Importing separate spaces

What we've got after running wikifetch.pl is one directory with all MediaWiki pages in it and all links pointing to the space MySpacePlaceholder.

First, we needed to decide which page goes where. There is no easy way to do this, so we just sorted them out manually, moving the page files to different directories. For example, all development-related stuff went into a directory named development, whereas all sales-related files went to the sales directory and so on.

We could now go ahead and import the directories one by one if it weren't for the backlinks. Say you've got a link from Main.WebHome to Development.Development. How would the import script know that Development is now located in the Development space?

We modified import.groovy to resolve the back links using a copy of the pages to be imported (the originals will get deleted as mentioned in part II).

First we'll need a hash map of the spaces:

[...]
def fileSpaces = [:];

new File( "C:/temp/copy_wiki" ).eachFileRecurse() {
  f->
  parentDir = f.parentFile.name;
  fileSpaces[ f.name ] = parentDir + "." + f.name;
}
[...]

What we've got now is a hash map where we can look up a "Space.Page" for a given "Page".

This we'll need in the main import loop:

[...]
		  fileAsText = f.getText();
		  
		  fileSpaces.each { 
				pageName, newName ->
		  		    fileAsText = fileAsText.replaceAll( "MySpacePlaceholder." + pageName, newName );
		  };
[...]

Which replaces all "MySpacePlaceholder.Page" with "Space.Page" where "Space" is the correct space-name. This could be optimized as it does a search replace n^2 times but for a one-time import this shouldn't matter much -- it wasn't much of a problem with our few hundred pages.

Now we're free to start the import, one directory at the time. The new version of import.groovy can be downloaded here.

Problems

There are still some unresolved problems regarding the export/import chain. One is that some pages ended up with garbled pre sections (the handling in XWiki is somewhat "unusual" by having no closing tags). Another one is that we haven't set the page's title automatically. This could have been the first h1 or h2 on the page. But apart from these problems and some special characters, it "worked" for us and we've since happily moved on with our XWiki.