Showing posts with label Ubuntu. Show all posts
Showing posts with label Ubuntu. Show all posts

Monday, January 17, 2011

Use Unison to Synchronize your remote shares

At the office and around the house, I often like to keep directories synchronized with network shares. Microsoft has provided two-way, remote folder sync for quite a while now. It is also possible to perform on Linux with a nifty utility named Unison.

Unison allows you to synchronize in both directions and builds on top of the tried and true rsync protocol. It's built to play well with file exchanges between Unix and Windows hosts. It also has a number of options that allow you to fine tune your sync or script the whole operation. There is a GUI version as well.

You can install it on Debian/Ubuntu with apt-get:
sudo apt-get install unison unison-gtk

In my daily use, I typically have several Nautilus .gvfs mounts to various Windows SMB/CIFS shares and SFTP hosts. Unison isn't directly aware of these Nautilus style mounts so I cobbled together this Nautilus script based on some examples I found at http://g-scripts.sourceforge.net.

Instructions

Copy the script to your ~/.gnome2/nautilus-scripts/ directory with the name unison-sync.sh.

Set the execute bit on the script.

Make sure zenity is installed.
sudo apt-get install zenity

With Nautilus, connect to a server resource using SMB or SFTP.

Right click on a remote directory and click scripts>unison-sync.sh.

A file directory dialog will appear. This allows you to select the local location you want to synchronize with the server.

Save the name of the Unison preference file.

Now run Unison from the terminal or the GUI.
unison pref_name 

Note

My script enable auto approve for non-conflicts to save time. You might want to change that. It also disables permissions since Windows mounts don't support the same types as standard Linux file systems.

The unison-sync.sh script:
#!/bin/bash 
#
#       This program is free software; you can redistribute it and/or modify
#       it under the terms of the GNU General Public License as published by
#       the Free Software Foundation; either version 2 of the License, or
#       (at your option) any later version.
#       
#       This program is distributed in the hope that it will be useful,
#       but WITHOUT ANY WARRANTY; without even the implied warranty of
#       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#       GNU General Public License for more details.
#       
#       You should have received a copy of the GNU General Public License
#       along with this program; if not, write to the Free Software
#       Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
#       MA 02110-1301, USA.
#
#  author :
#    clayton.kramer  gmail.com 
#
#  description :
#    Provides a quick way of making Unison preference files from
#    Nautilus.
#
#  informations :
#    - a script for use (only) with Nautilus. 
#    - to use, copy to your ${HOME}/.gnome2/nautilus-scripts/ directory.
#
#  WARNINGS :
#    - this script must be executable.
#    - package "zenity" must be installed
#
#  THANKS :
#    This script was heavily sourced from the work of SLK. Having
#    Perl regex to parse .gvfs paths was a huge time saver.
#    

# CONSTANTS

# some labels used for zenity [en]
z_title="Synchronize Folder"
z_err_gvfs="cannot acces to directory - check gvfs\nEXIT"
z_err_uri="cannot acces to directory - uri not known\nEXIT"

# INIT VARIABLES

# may depends of your system : (current settings for debian, ubuntu)

GVFSMOUNT='/usr/bin/gvfs-mount'
GREP='/bin/grep'
IFCONFIG='/sbin/ifconfig'
KILL='/bin/kill'
LSOF='/usr/bin/lsof'
PERL='/usr/bin/perl'
PYTHON='/usr/bin/python2.5'
SLEEP='/bin/sleep'
ZENITY='/usr/bin/zenity'

# MAIN

export LANG=C

# retrieve the first object selected or the current uri
if [ "$NAUTILUS_SCRIPT_SELECTED_URIS" == "" ] ; then
    uri_first_object=`echo -e "$NAUTILUS_SCRIPT_CURRENT_URI" \
      | $PERL -ne 'print;exit'`
else
    uri_first_object=`echo -e "$NAUTILUS_SCRIPT_SELECTED_URIS" \
      | $PERL -ne 'print;exit'`
fi

type_uri=`echo "$uri_first_object" \
  | $PERL -pe 's~^(.+?)://.+$~$1~'`

# try to get the full path of the uri (local path or gvfs mount ?)
if [ $type_uri == "file" ] ; then
    
    filepath_object=`echo "$uri_first_object" \
      | $PERL -pe '
        s~^file://~~;
        s~%([0-9A-Fa-f]{2})~chr(hex($1))~eg'`
    
elif [ $type_uri == "smb" -o $type_uri == "sftp" ] ; then
    if [ -x $GVFSMOUNT ] ; then
        
        # host (and share for smb) are matching a directory in ~/.gvfs/
        
        host_share_uri=`echo "$uri_first_object" \
          | $PERL -pe '
            s~^(smb://.+?/.+?/).*$~$1~;
            s~^(sftp://.+?/).*$~$1~;
            '`
        
        path_gvfs=`${GVFSMOUNT} -l  \
          | $GREP "$host_share_uri" \
          | $PERL -ne 'print/^.+?:\s(.+?)\s->.+$/'`
        
        # now let's create the local path
        path_uri=`echo "$uri_first_object" \
          | $PERL -pe '
            s~^smb://.+?/.+?/~~;
            s~^sftp://.+?/~~;
            s~%([0-9A-Fa-f]{2})~chr(hex($1))~eg'`
        
        filepath_object="${HOME}/.gvfs/${path_gvfs}/${path_uri}"
        
    else
        $ZENITY --error --title "$z_title" --width "320" \
          --text="$z_err_gvfs"
        
        exit 1
    fi
else
    $ZENITY --error --title "$z_title" --width "320" \
      --text="$z_err_uri"
    
    exit 1
fi


if [ -d "${HOME}/.unison" ]; then
    # create the Unison user directory if it doesn't exist
    mkdir -p "${HOME}/.unison"
fi

# Select a local directory to sync with
local_dir=`$ZENITY --title "$z_title" --file-selection --directory`

# Provide an alias for the sync
mount_name=`echo "$filepath_object" |  perl -ne 'print/main on (\w*)\//'`

base_name=`echo "$filepath_object" | perl -ne 'print/.*\/(.*)$/;'`
alias="$mount_name-$base_name"
alias=`$ZENITY --title "$z_title" --entry --text="Enter a name for this Unison preferences file." --entry-text="$alias"`
alias="$alias.prf"

# Write the Unison file
echo "# Unison preferences file" > ${HOME}/.unison/$alias
echo "root = $local_dir" >> ${HOME}/.unison/$alias
echo "root = $filepath_object" >> ${HOME}/.unison/$alias
echo "perms = 0" >> ${HOME}/.unison/$alias
echo "dontchmod = true" >> ${HOME}/.unison/$alias
echo "auto = true" >> ${HOME}/.unison/$alias

exit 0


### EOF

Sunday, January 16, 2011

Puppet manifest for Centrify Express on Ubuntu

I've been really pleased with Canonical's new partnership with Centrify, one of the big names in Unix/Linux/Mac Active Directory integration. For the last month, I've started to replace Likewise Open on all of our machines at work.

Tonight, I took a moment to write a quick Puppet manifest for installing centrifydc and automatically joining the machine to our AD infrastructure.

Requirements
  • Have an AD user account with privileges to add more than 10 computers to your domain.
  • Enable the Canonical partner repository (I manage my /etc/apt/sources.list with Puppet)
This script is going to expose a user account password in a text file so make sure you lock it down at same time you delegate the computer object permissions. (If anyone has a better way, I'd appreciate a comment from you.)

class centrify {

        package { centrifydc :
                ensure => latest ,
                notify => Exec["adjoin"]
        }

        exec { "adjoin" :
                path => "/usr/bin:/usr/sbin:/bin",
                returns => 15,
                command => "adjoin -w -u domainjoiner -p passwordF00 my.company.net",
                refreshonly => true,
        }

        service { centrifydc:
                ensure  => running
        }

}

The domain join action is only executed when Puppet detects that the package has to be installed or updated. Successful AD joins return a "15" code instead of the normal "0".

Tuesday, July 27, 2010

Grub_puts not found

Two of our Ubuntu 10.4 Lucid workstations ran into Grub2 errors today. Something must have gone wrong with the grub2 apt scripts while they were updating to the latest kernel. Both of the machines with the problem were created from the same Clonezilla image but a few of the other cloned machines weren't affected.

After running the apt-get dist-upgrade command and rebooting, my users encountered the "fix symbol 'grub_puts' not found" error.

Instructions

Burn the Ubuntu desktop ISO to CDROM or use the System > Administrator > Startup Disk Creator to create a bootable USB stick.

Boot from your live disk.

Open a terminal and get a list of the available partitions.

sudo fdisk -l

You should see results that look something like this:

Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e0719

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          32      248832   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              32        9730    77899777    5  Extended
/dev/sda5              32        9730    77899776   83  Linux

Disk /dev/sdb: 8053 MB, 8053063680 bytes
255 heads, 63 sectors/track, 979 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00009233

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1         255     2048256    b  W95 FAT32
/dev/sdb2             256         979     5815530    b  W95 FAT32

In my example above, you can see the system drive is listed as /dev/sda and the bootable USB is /dev/sdb. You may, like me, have a separate /boot partition because you are running encrypted LVM volumes. In that case you need to pay attention to which is your root volume.

Mount your "root" partition or volume first. Standard Linux partitions are simple.

sudo mount /dev/sda1 /mnt

An encrypted LVM is a little more complicated. The Ubuntu Live CD doesn't have the LVM crypto packages installed so run these commands to get it working.

sudo apt-get install lvm2 cryptsetup

Load the dm-crypt module.

sudo modprobe dm-crypte

Now unlock your encrypted volume. Enter your LUKS passphrase when prompted.

sudo cryptsetup luksOpen /dev/sda2 foo

Load the LVM Kernel module.

sudo modprobe dm-mod

Scan for all of the available volume groups.

sudo vgscan

Active the volume group.

sudo vgchange -a

Now list the logical volumes along with their /dev paths. In the example below, note that my laptop is named "falcon" and yours is most likely something else.

sudo lvscan
  ACTIVE            '/dev/falcon/root' [71.22 GiB] inherit
  ACTIVE            '/dev/falcon/swap_1' [3.07 GiB] inherit

Now mount the root volume to /mnt. Replace falcon to match your own results of the previous command.

sudo mount /dev/falcon/root /mnt

Chroot Prep

Now mount the /dev, /proc, /sys folders for os-prober and grub to work properly in a chrooted jail.

sudo mount --bind /dev /mnt/dev
sudo mount --bind /proc /mnt/proc
sudo mount --bind /sys /mnt/sys

If you had separate /boot partition because of LVM then mount it now.

sudo mount /dev/sda1 /mnt/boot

Now chroot yourself.

sudo chroot /mnt

Repair Grub2

Run the grub-mkconfig command to generate a new grub2 configuration file. This might be what got corrupted and left in this lurch.

grub-mkconfig -o /boot/grub/grub.cfg

Make sure no errors were generated. Then install grub2 in the hard drive MBR.

grub-install /dev/sda

Again make sure didn't get any errors. If you want a warm and fuzzy test your repair with the recheck option.

grub-install --recheck /dev/sda

Exit out of chroot with an exit or Crt+D command.

Unmount the directories.

sudo umount /mnt/dev
sudo umount /mnt

Now reboot and you should have your system back.

Monday, May 3, 2010

Use Clonezilla for physical disk to iSCSI volume transfer

For the last few nights, I've been playing around with open-iscsi on Debian, Ubuntu and Windows 2008. Getting things up and running was fairly straight forward thanks to all of the helpful blogs and howtos people have posted. What I found missing was how one moves a Linux installation from a physical or virtual disk to an iSCSI volume. The little I found about the subject involved physically mounting the source disk to the iSCSI host or performing some tricky PXE boot magic to run the Linux distribution's installer. I find both of these methods inelegant and limited.

Tonight I came at it again. This time with my favorite FOSS disk imaging tool, Clonezilla!. The wonderful team behind it didn't skimp out and included the open-iscsi packages.

Instructions


Download and burn a copy of the latest Ubuntu version of Clonezilla.

Boot from the Clonezilla live CDROM. Select all of the regional configuration options you require.

Stop when you get the the ncurses prompt to begin using Clonezilla or use the console. Press +F2 to switch the second tty console. This will let you work with the tools and setup a connection to your iSCSI share.

Get some networking configured otherwise you aren't going to be able to connect to the LUN.
sudo dhclient eth0

Now edit the iscsid.conf file.
sudo vi /etc/iscsi/iscsid.conf

Look for the node.startup property and set it to automatic.

Now start the open-iscsi daemon.
sudo /etc/init.d/open-iscsi start

Use the following command to query the iSCSI target for LUNs.
iscsiadm -m discovery -t sendtargets -p IP_OF_YOUR_TARGET

Here's an example of what mine looked like:
user@karmic:~$ sudo iscsiadm -m discovery -t st -p localhost
192.168.50.10:3260,1 iqn.2007-10.local.server-1:storage.lun0

Now I can connect using the following:
iscsiadm -m node -T iqn.2007-10.local.server-1:storage.lun0 -p 192.168.50.10:3260 -l

Now check the /var/log/messages for the newly created virtual SCSI device.
tail /var/log/messages

Now you can switch back to console #1 and continue with Clonezilla wizard. Select local disk to local disk when prompted for which mode to use.

Create Cisco VPN on Ubuntu Karmic/Lucid

It is very easy to setup a Cisco VPN on Ubuntu. I used the following instructions to get my corporate tunnels running. This tutorial assumes you have already acquired a .pcf file from your network IT staff.

Instructions

Install the vpnc package and any required dependencies:
sudo apt-get install vpnc

Open your vpn pcf configuration file with your favorite text editor.
vim corporatenet.pcf

It will looking something like this:
[main]
Description=
Host=vpn.corpnet.com
AuthType=1
GroupName=CorpNet
GroupPwd=enc_GroupPwd=C555E3A4BE82FF0001601A38260A92D93FF5693A482367E117EF8697CBED681C5FDD7F2AE0DEEA4B37DBBB21434189A46D8955F11916040A
EnableISPConnect=0
ISPConnectType=0
ISPConnect=
ISPPhonebook=
ISPCommand=
Username=
SaveUserPassword=0
UserPassword=
enc_UserPassword=
NTDomain=
EnableBackup=0
BackupServer=
EnableMSLogon=1
MSLogonType=0
EnableNat=1
TunnelingMode=0
TcpTunnelingPort=10000
CertStore=0
CertName=
CertPath=
CertSubjectName=
CertSerialHash=00000000000000000000000000000000
SendCertChain=0
PeerTimeout=90
EnableLocalLAN=0

Note the values for Host, GroupName and enc_GroupPwd. You'll need these to create your vpnc configuration file.

sudo vim /etc/vpnc/corpnet.conf

Make your configuration file look like this. Just make sure to change the fictional CorpNet values with your own.

IPSec gateway vpn.corpnet.com
IPSec ID CorpNet
IPSec obfuscated secret C555E3A4BE82FF0001601A38260A92D93FF5693A482367E117EF8697CBED681C5FDD7F2AE0DEEA4B37DBBB21434189A46D8955F11916040A
Xauth username YOURUSERNAME
Xauth password YOURPASSWORD

It's important to note the obfuscated option in the group password. Most of the examples and howtos I've seen on the Net leave this out because they were written several years ago before VPNC supported Cisco encrypted passwords. The older guides required you to de-crypt the Cisco string. This isn't necessary anymore with Karmic and Lucid releases.

Thursday, April 29, 2010

Run Windows virtual machines on Ubuntu/Debian desktop with KVM

Both at home and at work, I use Ubuntu as my operating system. There are times when I'm forced use Windows for some reason and there are several solutions for host Windows OS virtual machines on an Ubuntu laptop. Several years ago, I used what I most understood, VMware's workstation offering for Linux. Later when Virtualbox-ose (open source edition) caught up with VMware's features and hosted from Ubuntu's repositories, I switched to it.

These days, I'm much more technically adept with FOSS virtualization technologies and made the switch to using Linux KVM on my newer machines which support Intel's VT and AMD's AMD-V acceleration. I don't have any Phoronix style detailed comparisons but KVM feels faster and lighter than Virtualbox or VMware.

Quick Setup

Install the qemu-kvm package
sudo apt-get install qemu-kvm

Create a directory to hold your virtual machines.
mkdir -p ~/VM/WinXP

Move to that directory and create a disk image file.
cd ~/VM/WinXP
qemu-img create -f raw windows_xp.img 12G

Options:
-f raw = creates raw IO driver format image (You could also use the qcow2 mode. It has more features but doesn't perform as fast as raw)
windows_xp.img = name of the image file
12G = The virtual disk size.

Now create a bash script using your favourite text editor. I like vim but you could just as easily use gedit from GNOME.
vim Windows_XP.sh

Here's how my script looks:
#!/bin/bash
#
# Description: Launches Windows XP QEMU64
#
# Verion: 1
# Author: Clayton Kramer clayton.kramer @ gmail.com
# Modified: Fri 23 Apr 2010 11:43:35 AM EDT 
#

# Ubuntu Karmic tweek - Prepare audio to use Pulse driver instead of ALSA 
export QEMU_AUDIO_DRV=pa

# Launch Windows XP KVM
kvm  \
    -name "Windows XP Guest" \
    -m 1024 \
    -smp 1 \
    -localtime \
    -drive file=~/VM/WinXP/windows_xp.img,if=virtio,index=0,boot=on,cache=writeback \
    -drive file=~/ISO/windows_xp_sp2.iso,if=ide,media=cdrom,index=2 \
    -fda ~/ISO/viostor-31-03-2010-floppy.img \
    -net nic,model=virtio \
    -net user \
    -soundhw ac97 \
    -usb \
    -usbdevice tablet 


By default Ubuntu 9.10's qemu-kvm will use ALSA drivers which can lead to some choppy sound. You can change this behavior by setting the QEMU_AUDIO_DRV environmental variable to pa before launching the KVM.

I am using the VirtIO drivers in the script above. They improve the IO performance for Windows guests. Haydn Solomon provides some detailed instructions on setting them up in his KVM blog. I've decided to live a little dangerous and enabled the writeback option for the block driver.

http://www.linux-kvm.com/content/block-driver-updates-install-drivers-during-windows-installation

After the Windows installation is complete you can ommit the virtual floppy disk device line.

You may also want to take note that my script also configures the paravirtualized network device. You'll need to get the latest driver for that from:

http://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers

If you wanted wanted to get a Windows XP install going without using the VirtIO drivers you can use this compatibility script. It uses IDE for the IO controller bus and Intel e1000 driver for the NIC.

# Launch Windows XP KVM (compatibility)
kvm  \
    -name "Windows XP Guest" \
    -m 1024 \
    -smp 1 \
    -localtime \
    -drive file=~/VM/WinXP/windows_xp.img,if=ide,index=0,boot=on \
    -drive file=~/ISO/windows_xp_sp2.iso,if=ide,media=cdrom,index=2 \
    -net nic,model=e1000 \
    -net user \
    -soundhw ac97 \
    -usb \
    -usbdevice tablet 

Monday, April 26, 2010

How do you manage multiple Ubuntu desktops?

I spent a large part of my day at work trying to figure out how to replicate Windows style login scripts for our office Ubuntu desktops. This seemed like a straight forward problem that some amount of community effort would have already solved. It was so easy to setup Active Directory (AD) integration with Likewise-open so where is the howto on setting up automatic drive mapping on a Gnome desktop?

There are mass management tools like CFGengine and Puppet for server farms but where are the tools for running an office on Linux desktops? There are some proprietary offerings from Likewise (the contributers of Likewise-open) and Centrify that provide tools for integrating AD group policy objects (GPO) but they are geared toward fortune 2000 size companies. I'm looking for something beyond just being able to authenticate with a AD and connect to a CIFS share. If Linux and especially Ubuntu are ever going to really crack the desktop market, someone needs to launch a project to bridge this small enterprise gap.