Wednesday, August 4, 2010

Ubuntu 10.04 MySQL Server startup bug

I encountered an issue yesterday with the mysql-server package on a server when I attempted to use the my.large.cnf settings file in place of the default.

See Launchpad: https://bugs.launchpad.net/ubuntu/+source/mysql-dfsg-5.1/+bug/566736

There is a bug with Ubuntu 10.04's MySQL server. If you have it bind to a specific interface, it will fail to start on reboot because it attempts to start after any network interface (such as 127.0.0.1) is initialized. If the interface that MySQL is bound to isn't reinitialized, it will hang. If you try to remove any specific interface bindings from the my.cnf settings, you'll run into another problem port assignment. You need to make sure that the upstart init script matches the bind-address value in the my.cnf file.

My edits:

/etc/mysql/my.cnf
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address            = 127.0.0.1
#

/etc/init/mysql.conf
start on (net-device-up IFACE=lo
          and local-filesystems
          and runlevel [2345])
stop on runlevel [016]

Note the IFACE=lo addition to the start on line.

My thanks to cdenley's post on the Ubuntu forums for shedding light on the problem.

Tuesday, July 27, 2010

Grub_puts not found

Two of our Ubuntu 10.4 Lucid workstations ran into Grub2 errors today. Something must have gone wrong with the grub2 apt scripts while they were updating to the latest kernel. Both of the machines with the problem were created from the same Clonezilla image but a few of the other cloned machines weren't affected.

After running the apt-get dist-upgrade command and rebooting, my users encountered the "fix symbol 'grub_puts' not found" error.

Instructions

Burn the Ubuntu desktop ISO to CDROM or use the System > Administrator > Startup Disk Creator to create a bootable USB stick.

Boot from your live disk.

Open a terminal and get a list of the available partitions.

sudo fdisk -l

You should see results that look something like this:

Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e0719

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          32      248832   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              32        9730    77899777    5  Extended
/dev/sda5              32        9730    77899776   83  Linux

Disk /dev/sdb: 8053 MB, 8053063680 bytes
255 heads, 63 sectors/track, 979 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00009233

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1         255     2048256    b  W95 FAT32
/dev/sdb2             256         979     5815530    b  W95 FAT32

In my example above, you can see the system drive is listed as /dev/sda and the bootable USB is /dev/sdb. You may, like me, have a separate /boot partition because you are running encrypted LVM volumes. In that case you need to pay attention to which is your root volume.

Mount your "root" partition or volume first. Standard Linux partitions are simple.

sudo mount /dev/sda1 /mnt

An encrypted LVM is a little more complicated. The Ubuntu Live CD doesn't have the LVM crypto packages installed so run these commands to get it working.

sudo apt-get install lvm2 cryptsetup

Load the dm-crypt module.

sudo modprobe dm-crypte

Now unlock your encrypted volume. Enter your LUKS passphrase when prompted.

sudo cryptsetup luksOpen /dev/sda2 foo

Load the LVM Kernel module.

sudo modprobe dm-mod

Scan for all of the available volume groups.

sudo vgscan

Active the volume group.

sudo vgchange -a

Now list the logical volumes along with their /dev paths. In the example below, note that my laptop is named "falcon" and yours is most likely something else.

sudo lvscan
  ACTIVE            '/dev/falcon/root' [71.22 GiB] inherit
  ACTIVE            '/dev/falcon/swap_1' [3.07 GiB] inherit

Now mount the root volume to /mnt. Replace falcon to match your own results of the previous command.

sudo mount /dev/falcon/root /mnt

Chroot Prep

Now mount the /dev, /proc, /sys folders for os-prober and grub to work properly in a chrooted jail.

sudo mount --bind /dev /mnt/dev
sudo mount --bind /proc /mnt/proc
sudo mount --bind /sys /mnt/sys

If you had separate /boot partition because of LVM then mount it now.

sudo mount /dev/sda1 /mnt/boot

Now chroot yourself.

sudo chroot /mnt

Repair Grub2

Run the grub-mkconfig command to generate a new grub2 configuration file. This might be what got corrupted and left in this lurch.

grub-mkconfig -o /boot/grub/grub.cfg

Make sure no errors were generated. Then install grub2 in the hard drive MBR.

grub-install /dev/sda

Again make sure didn't get any errors. If you want a warm and fuzzy test your repair with the recheck option.

grub-install --recheck /dev/sda

Exit out of chroot with an exit or Crt+D command.

Unmount the directories.

sudo umount /mnt/dev
sudo umount /mnt

Now reboot and you should have your system back.

Wednesday, June 2, 2010

Blurry Linux KVM screen fix

QEMU/KVM scales the guest's screen to fit a re-sized window. This is one of the small "paper cuts" that I've been living with since moving to KVM for my virtualization needs.



Having the screen get resized fine for occasions where you want to lessen the screen real-estate of a VM but still keep an eye want is going on. The problem is once you try drag the window back to it's 1:1 size. With a free hand you won't be able to get the window size exactly back to a 1:1 ratio so the everything in the VM will look slightly blurry.

Here's an example of my trying to get this 1152 x 864 Windows XP KVM back to actual screen resolution. It's close but still blurry.

I've lived with this blurriness for months now but today I came across Al Dimond's post about the problem. He took the time to investigate the problem and found a quick workaround using xdotool to resize the KVM window to a width and height one pixel less than the guest.

First get the window ID of your KVM
xdotool search --title QEMU

Then use the windowsize option to set the window to an exact size. The window ID in my example is 90177539.

xdotool windowsize 90177539 1151 863

Following the one pixel less workaround, you would use this command for a 1024x768 guest.

xdotool windowsize 90177539 1023 767

Now the guest screen is sharp and crisp again.

Wednesday, May 5, 2010

Crank up the throttle on DD transfers

This is a quick one but I want to write it down just because I always seem to forget it if I don't use DD for a while.

DD's default block size (bs) is 512 bytes!

That's fine for some small work but you'll be waiting around for hours if your trying to shovel large disk images around.

Kick DD into high gear by raising the bs value to 32k and be done in minutes.

Example:
dd if=/dev/loop0 of=/dev/sdb bs=32k

In the example above, I've mounted a KVM disk image locally and an iSCSI lun as /dev/sdb. With the default 512b, DD was only able capitalize my 1g network at 5Mb/s. Switching to a 32k block size it utilized 40Mb/s.

Monday, May 3, 2010

Use Clonezilla for physical disk to iSCSI volume transfer

For the last few nights, I've been playing around with open-iscsi on Debian, Ubuntu and Windows 2008. Getting things up and running was fairly straight forward thanks to all of the helpful blogs and howtos people have posted. What I found missing was how one moves a Linux installation from a physical or virtual disk to an iSCSI volume. The little I found about the subject involved physically mounting the source disk to the iSCSI host or performing some tricky PXE boot magic to run the Linux distribution's installer. I find both of these methods inelegant and limited.

Tonight I came at it again. This time with my favorite FOSS disk imaging tool, Clonezilla!. The wonderful team behind it didn't skimp out and included the open-iscsi packages.

Instructions


Download and burn a copy of the latest Ubuntu version of Clonezilla.

Boot from the Clonezilla live CDROM. Select all of the regional configuration options you require.

Stop when you get the the ncurses prompt to begin using Clonezilla or use the console. Press +F2 to switch the second tty console. This will let you work with the tools and setup a connection to your iSCSI share.

Get some networking configured otherwise you aren't going to be able to connect to the LUN.
sudo dhclient eth0

Now edit the iscsid.conf file.
sudo vi /etc/iscsi/iscsid.conf

Look for the node.startup property and set it to automatic.

Now start the open-iscsi daemon.
sudo /etc/init.d/open-iscsi start

Use the following command to query the iSCSI target for LUNs.
iscsiadm -m discovery -t sendtargets -p IP_OF_YOUR_TARGET

Here's an example of what mine looked like:
user@karmic:~$ sudo iscsiadm -m discovery -t st -p localhost
192.168.50.10:3260,1 iqn.2007-10.local.server-1:storage.lun0

Now I can connect using the following:
iscsiadm -m node -T iqn.2007-10.local.server-1:storage.lun0 -p 192.168.50.10:3260 -l

Now check the /var/log/messages for the newly created virtual SCSI device.
tail /var/log/messages

Now you can switch back to console #1 and continue with Clonezilla wizard. Select local disk to local disk when prompted for which mode to use.

Create Cisco VPN on Ubuntu Karmic/Lucid

It is very easy to setup a Cisco VPN on Ubuntu. I used the following instructions to get my corporate tunnels running. This tutorial assumes you have already acquired a .pcf file from your network IT staff.

Instructions

Install the vpnc package and any required dependencies:
sudo apt-get install vpnc

Open your vpn pcf configuration file with your favorite text editor.
vim corporatenet.pcf

It will looking something like this:
[main]
Description=
Host=vpn.corpnet.com
AuthType=1
GroupName=CorpNet
GroupPwd=enc_GroupPwd=C555E3A4BE82FF0001601A38260A92D93FF5693A482367E117EF8697CBED681C5FDD7F2AE0DEEA4B37DBBB21434189A46D8955F11916040A
EnableISPConnect=0
ISPConnectType=0
ISPConnect=
ISPPhonebook=
ISPCommand=
Username=
SaveUserPassword=0
UserPassword=
enc_UserPassword=
NTDomain=
EnableBackup=0
BackupServer=
EnableMSLogon=1
MSLogonType=0
EnableNat=1
TunnelingMode=0
TcpTunnelingPort=10000
CertStore=0
CertName=
CertPath=
CertSubjectName=
CertSerialHash=00000000000000000000000000000000
SendCertChain=0
PeerTimeout=90
EnableLocalLAN=0

Note the values for Host, GroupName and enc_GroupPwd. You'll need these to create your vpnc configuration file.

sudo vim /etc/vpnc/corpnet.conf

Make your configuration file look like this. Just make sure to change the fictional CorpNet values with your own.

IPSec gateway vpn.corpnet.com
IPSec ID CorpNet
IPSec obfuscated secret C555E3A4BE82FF0001601A38260A92D93FF5693A482367E117EF8697CBED681C5FDD7F2AE0DEEA4B37DBBB21434189A46D8955F11916040A
Xauth username YOURUSERNAME
Xauth password YOURPASSWORD

It's important to note the obfuscated option in the group password. Most of the examples and howtos I've seen on the Net leave this out because they were written several years ago before VPNC supported Cisco encrypted passwords. The older guides required you to de-crypt the Cisco string. This isn't necessary anymore with Karmic and Lucid releases.

Thursday, April 29, 2010

Run Windows virtual machines on Ubuntu/Debian desktop with KVM

Both at home and at work, I use Ubuntu as my operating system. There are times when I'm forced use Windows for some reason and there are several solutions for host Windows OS virtual machines on an Ubuntu laptop. Several years ago, I used what I most understood, VMware's workstation offering for Linux. Later when Virtualbox-ose (open source edition) caught up with VMware's features and hosted from Ubuntu's repositories, I switched to it.

These days, I'm much more technically adept with FOSS virtualization technologies and made the switch to using Linux KVM on my newer machines which support Intel's VT and AMD's AMD-V acceleration. I don't have any Phoronix style detailed comparisons but KVM feels faster and lighter than Virtualbox or VMware.

Quick Setup

Install the qemu-kvm package
sudo apt-get install qemu-kvm

Create a directory to hold your virtual machines.
mkdir -p ~/VM/WinXP

Move to that directory and create a disk image file.
cd ~/VM/WinXP
qemu-img create -f raw windows_xp.img 12G

Options:
-f raw = creates raw IO driver format image (You could also use the qcow2 mode. It has more features but doesn't perform as fast as raw)
windows_xp.img = name of the image file
12G = The virtual disk size.

Now create a bash script using your favourite text editor. I like vim but you could just as easily use gedit from GNOME.
vim Windows_XP.sh

Here's how my script looks:
#!/bin/bash
#
# Description: Launches Windows XP QEMU64
#
# Verion: 1
# Author: Clayton Kramer clayton.kramer @ gmail.com
# Modified: Fri 23 Apr 2010 11:43:35 AM EDT 
#

# Ubuntu Karmic tweek - Prepare audio to use Pulse driver instead of ALSA 
export QEMU_AUDIO_DRV=pa

# Launch Windows XP KVM
kvm  \
    -name "Windows XP Guest" \
    -m 1024 \
    -smp 1 \
    -localtime \
    -drive file=~/VM/WinXP/windows_xp.img,if=virtio,index=0,boot=on,cache=writeback \
    -drive file=~/ISO/windows_xp_sp2.iso,if=ide,media=cdrom,index=2 \
    -fda ~/ISO/viostor-31-03-2010-floppy.img \
    -net nic,model=virtio \
    -net user \
    -soundhw ac97 \
    -usb \
    -usbdevice tablet 


By default Ubuntu 9.10's qemu-kvm will use ALSA drivers which can lead to some choppy sound. You can change this behavior by setting the QEMU_AUDIO_DRV environmental variable to pa before launching the KVM.

I am using the VirtIO drivers in the script above. They improve the IO performance for Windows guests. Haydn Solomon provides some detailed instructions on setting them up in his KVM blog. I've decided to live a little dangerous and enabled the writeback option for the block driver.

http://www.linux-kvm.com/content/block-driver-updates-install-drivers-during-windows-installation

After the Windows installation is complete you can ommit the virtual floppy disk device line.

You may also want to take note that my script also configures the paravirtualized network device. You'll need to get the latest driver for that from:

http://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers

If you wanted wanted to get a Windows XP install going without using the VirtIO drivers you can use this compatibility script. It uses IDE for the IO controller bus and Intel e1000 driver for the NIC.

# Launch Windows XP KVM (compatibility)
kvm  \
    -name "Windows XP Guest" \
    -m 1024 \
    -smp 1 \
    -localtime \
    -drive file=~/VM/WinXP/windows_xp.img,if=ide,index=0,boot=on \
    -drive file=~/ISO/windows_xp_sp2.iso,if=ide,media=cdrom,index=2 \
    -net nic,model=e1000 \
    -net user \
    -soundhw ac97 \
    -usb \
    -usbdevice tablet