Upgrading Ubuntu at work can make you rather unpopular, as the Internet bandwidth is fully utilised downloading all the updates to packages you have long since forgotten that you installed.

It also takes time, time that you should be working rather than upgrading your computer.

For these reasons I like to trickle download the upgrade over a day and only perform the actual upgrade once all the packages are ready, typically the following morning.

This is how I performed my low-bandwidth upgrade…

N.B. This is not the official or recommended way of upgrading between Ubuntu versions. Specifically my method involves manually disabling some repositories and updating others to the new release. This would normally be done by the do-release-upgrade program itself. It works for me, but please do be aware that you are deviating slightly from the official method.

Step 1: Disable 3rd Party Repositories

Launch the Ubuntu Software Centre and from the menu select Edit followed by Software Sources. Under the Other Software tab please untick all active repositories.

(This step should in any case be done automatically by step 4).

Step 2: Update Repositories

Edit /etc/apt/sources.list and replace all occurrences of  ”precise” with “trusty”. If you are of a brave disposition, the following command should do this for you:

$ sudo sed -i.bak 's/precise/trusty/g' sources.list

(This will create a copy of sources.list to sources.list.bak, in case you wish to reverse this.)

Step 3: Download Packages

Still in the terminal, type:

$ sudo apt-get updatesudo apt-get -o Acquire::http::Dl-Limit=64 -d dist-upgrade

The 64 will limit the bandwidth to 64 Kbps, please adjust to suit your available bandwidth. The “-d” will instruct apt-get to merely download the packages and not to install them.

I believe this stage can be aborted with Ctrl+C at any time and run again, until such time as all the required packages are downloaded.

Step 4: Upgrade

Still in the terminal, I tend to use GNU Screen for extra resilience, type:

$ sudo do-release-upgrade

Conclusion

I am typing this on my newly upgraded 14.04 installation, after a clean and trouble-free reboot and an entirely fault-free upgrade.

The truly astonishing aspect to an upgrade is the fact that the computer remains largely usable throughout. I lost my fonts briefly in one application during Step 4, but otherwise I was able to work normally. It didn’t even seem to be slowing my computer down greatly, although this is a fairly powerful workhorse, so your mileage may vary.

Please do comment, if you feel I’ve left anything out in the above, or indeed if you have found it useful.

Good luck with your upgrade.

I have long been a keen user of pdftk, the PDF Toolkit, but am frequently surprised when people have not heard of it. True, it is a command line tool, but it is easy to incorporate into service menus, scripts etc and doubtless there is a GUI front-end for it somewhere (in fact there is one linked to from the above page).

Clearly a blog post is called for, but, whilst you wait for a post that will never arrive, here is a link to some examples that should open your eyes to what is possible with pdftk.

To get started on a Debian-based system:

$ sudo apt-get install pdftk
$ man pdftk

Enjoy.

Very early on in my Linux life, I came across this suggested header for crontab and I’ve used it ever since. So much so that I am always slightly thrown when I come across a crontab without it! No, you don’t need it, yes the standard commented header works just fine, but, if like me you prefer things neatly lined up, then this might suit you:

MAILTO=
#   _________________________ 2. Minute - Minutes after the hour (0-59)
#  |
#  |      ______________________ 2. Hour - 24-hour format (0-23).
#  |     |
#  |     |      ___________________ 3. Day - Day of the month (1-31)
#  |     |     |
#  |     |     |      ________________ 4. Month - Month of the year (1-12)
#  |     |     |     |
#  |     |     |     |     ______________5. Weekday - Day of the week. (0-6, 0 indicates Sunday)
#  |     |     |     |    |
#__|_____|_____|_____|____|___Command_____________________________________________________________

And if you recognise this as your’s, then thank you!

For more information please visit the following link:

• HowTo: Make umount Work with sshfs | Unicom Systems Development

For more information please visit:

• Linux Terminal Command Reference

XPS is Microsoft’s attempt to replace PDF, the only difference is that everyone can read PDFs, and not everyone can read XPS. I understand that KDE 4 versions of Okular will support XPS, which may make these instructions unnecessary, although having a tool for conversion readily at hand is always useful!

These instructions were tested in Debian Lenny. These instructions worked for our specific systems YMMV.

Building from source

$ sudo apt-get install libxext-dev libxt-dev
$ wget http://ghostscript.com/releases/ghostpdl-8.71.tar.bz2
$ tar xvvjf ghostpdl-8.71.tar.bz2
$ cd ghostpdl-8.71
$ make xps

Testing

After the build you will find gxps in xps/obj

To test, you will need a test document in XPS format.

$ cd xps/obj
$ ./gxps -sDEVICE=pdfwrite -sOutputFile=test.pdf -dNOPAUSE test.xps

Move to bin

You probably want to move the gxps executable into a convenient location within your PATH. /usr/local/bin may be a good destination. Once there you ought to be able to run the command from anywhere and it just work. Not sure what your PATH is? Type “echo $PATH” in your terminal.

$ echo $PATH
$ sudo cp xps/obj/gxps /usr/local/bin/
$ sudo chown root:root /usr/local/bin/gxps

Creating file type

XPS probably does not exist on your Linux system as a file type, you can either create yourself using KDE Control Centre, or in KDE:

Single user:
~/.kde/share/mimelnk/application/xps.desktop

All users:
/usr/share/mimelnk/application/xps.desktop

[Desktop Entry]
Comment=XPS Document
Hidden=false
Icon=application-xps
MimeType=application/xps
Patterns=*.xps;*.XPS
Type=MimeType
X-KDE-AutoEmbed=false

Adding to ServiceMenu

If you are using Konqueror, you can add a service menu (to enable right-click / action menu).

Single user:
~/.kde/share/apps/konqueror/servicemenus/xpstopdf.desktop

All users:
/usr/share/apps/konqueror/servicemenus/xpstopdf.desktop

[Desktop Entry]
Version=1.0
Encoding=UTF-8
Name=xpstopdf service menu
ServiceTypes=application/xps
Icon=acroread
Actions=xpstopdf

[Desktop Action xpstopdf]
Icon=acroread
Name=Convert XPS to PDF
Exec=cd "%d"; gxps -sDEVICE=pdfwrite -sOutputFile="`echo "%f" | cut -d . -f 1`.pdf" -dNOPAUSE "%f"; mv "%f" ~/.local/share/Trash/files; kdialog --title "Convert XPS to PDF" --passivepopup "Done" 3; echo;

Testing ServiceMenu

You should now be able to right-click on the file and “Convert XPS to PDF”. This will create a PDF of the same name and move the XPS into trash.

References

• http://www.ghostscript.com/GhostPCL.html
• http://obscured.info/2010/03/01/converting-xps-to-pdf-on-ubuntu-9-10/
• http://blog.rubypdf.com/2009/04/14/convert-xps-to-pdf-in-two-ways/

# update-alternatives --config x-www-browser

Unfortunately of course many of the applications that I am using are native to KDE or Gnome and probably are still respecting their environment’s settings. In the end it was simply a case of editing:

~/.local/share/applications/defaults.list

And adding the following lines:

x-scheme-handler/http=chromium.desktop
x-scheme-handler/https=chromium.desktop

Now opening links from my terminal is correctly opening a new tab in Chromium, or running Chromium if it isn’t already.

Joy.

I have bought myself a GPS Receiver BU-353. Having plugged in the device into my Debian Wheezy workstation, I wanted to test that it was working.

A quick dmesg | tail showed me that the device has been found and installed correctly (no drivers required).

I then installed the GPS daemon:

$ sudo apt-get install gpsd gpsd-clients
$ sudo dpkg-reconfigure gpsd

This then started the GPS daemon. The next thing to do was get some example output, and the tool for this is gpspipe:

$ gpspipe -w -n 5

Lastly, I thought it would be fun to plot the output onto Google Maps and/or Openstreetmaps:

$ tpv=$(gpspipe -w -n 5 | grep -m 1 TPV | cut -d, -f4,6-8,13)
$ latitude=$(echo $tpv | cut -d, -f3 | cut -d: -f2)
$ longitude=$(echo $tpv | cut -d, -f4 | cut -d: -f2)
$ google_map_url="http://maps.google.com/?q=${longitude},${latitude}&z=${zoom}"
$ osm_map_url="http://www.openstreetmap.org/?mlat=${latitude}&mlon=${longitude}&zoom=${zoom}&layers=M"
$ xdg-open $google_map_url
$ xdg-open $osm_map_url

All worked beautifully.

Before you start, you need to have sane installed, and you also need tesseract-ocr - both should be available in your distros repositories.

$ sudo apt-get install sane-utils tesseract-ocr

Next you need to find out what scanners you have available, and you do this with:

$ scanimage -L
device `v4l:/dev/video0' is a Noname Vimicro USB Camera (Altair) virtual device
device `plustek:libusb:004:002' is a Epson Perfection 1250/Photo flatbed scanner)

Obviously the latter is my scanner.

Assuming you have a working scanner, the following is a simple two liner to scan and OCR.

$ scanimage -d 'plustek:libusb:004:002' --mode Lineart \
--format tiff -x 215 -y 297 --resolution 200 > example.tif

And finally convert to text with tesseract:

$ tesseract /tmp/example.tif example

You should now have a file example.txt in your current directory, which you can open in any text editor.

Obviously this has limitations - it works for single-page A4 portrait typed documents - but it gives you the basics.

You could probably experiment with the resolution, 200 worked for me, so I didn’t bother trying anything else.  Traditionally the higher the resolution the better, but I seem to recall that tesseract works better on 300 and below.

On my Epson Perfection 1250 I found that I needed to add the sane switch --warmup-time 0 as otherwise it never finished warming up.

If you would prefer to OCR an existing PDF, which is another thing that I find myself doing from time to time, then first convert it to a tif:

$ convert -density 200 example.pdf -depth 8 /tmp/example.tif

And then run the above tesseract command.

We all know that feeling when our disk fills up and you are left desperately scrabbling around to find out where your disk space has gone. In a previous blog post I discussed the use of the wonderful Konqueror File Size View, but this is no good for remote servers. Normally I would resort to “du” or the wonderful “find” utility to look for large files, but here is an interesting alternative that I had not come across before: ncdu (ncurses disk usage).

Its name tells you pretty much everything you need to know. It can be installed with a simple apt-get install ncdu and then the man page is a useful guide. In simple terms it can just be run with:

# ncdu /var

The lovely thing about ncdu is that once it completes its run (which can take a long time on a large disk or a nfs share), you can drill into the directory structure following the disk usage to determine where your space has gone.

It is a very simple program but one that I will find most useful.

apt-get

For apt-get you just need to create a new file:

/etc/apt/apt.conf.d/76download

with the following contents:

$ sudo vim /etc/apt/apt.conf.d/76download 
Acquire {
    Queue-mode "access";
    http {
        Dl-Limit "128";
    };
};
$

The above will limit your bandwidth to 128K, adjust this figure to suit your network.

Alternatively, if you don’t want this change to be set permanently, then you can specify it in the command line:

$ sudo apt-get -o Acquire::http::Dl-Limit=128 upgrade

wget

To rate-limit wget, simply edit /etc/wgetrc or your personal configuration at ~/.wgetrc and add or edit the following line:

limit-rate=128k

other

Other packages can be configured in different ways, but you could install trickle and then read its man page to determine how to use it. For example (from the man page):

$ trickle -u 128 -d 128 ncftp

This doesn’t give you the entire picture, as it only lists the dependencies that you don’t already have; which is usually all you care about, but there are occasions when you would like to list all of a package’s dependencies, for example when planning for a system that is not built yet, or not accessible at the current time. Or just for idle curiosity! Perhaps that’s just me.

Anyhow, apt-rdepends is the application for the job. It doesn’t just list the package’s dependencies, but it recursively goes through each dependency’s dependencies.

Install it with the usual $ sudo apt-get install apt-rdepends and then simply run with:

$ apt-rdepends packagename | less

Yes, it is quite verbose, hence the “| less” - leave it out if you prefer, or use “| more” which is more likely to be installed on your system (tip: install “less” - less is better than more, if that makes any sense).

For example, I had just installed “flite” and was amazed at how functional it was. I wondered to myself whether it was just a front-end to festival - but how to find out?

$ apt-rdepends flite

Which comes back with no other speech synthesis engine (e.g. festival), so clearly flite is a speech synthesis engine in its own right.

Linux.com :: Automatically process new files with fsniper

Now I am wondering if I can use it to prompt an rsync to sync our shared documents to our remote site, and it seems I could.  This is a major headache for me, as we have two branches and a shared documents repository.

I have previously tried using Unison to synchronise between the branches, but this has created a lot of load on the master server, massively slowing down performance for all the users on the master server.

We are currently using apache2+webdav+svn but this is not working well for us.  Potentially fsniper+rsync could work very well.  It would clearly need to be running on both servers and I can see a potential clash if the two servers happen to try and update the same file at the same time.  More thought is required, any suggestions - do let me know.

The links to the fsniper site seem to be outdated in both of the above articles, but it may be found at:

http://freshmeat.net/projects/fsniper

It doesn’t appear to be packaged for Debian, but I will probably compile from source and try it soon.  More later!

I am a rank amateur at both tar and mt. This page constitutes no more than you could discover yourself by reading the manpages for tar and mt, or Googling.

You have been warned!

Simple instructions

Rewind

Use the rewind command before backup to ensure that you are overwriting previous backups.

# mt -f /dev/st0 rewind

Backup

E.g. directory /www and /home with tar command (z - compressed)

# tar -czf /dev/st0 /www /home

Use -v to receive verbose feedback.

Backup with Verify

If you do not compress your backup, then you can verify in the same process:

# tar -cWf /dev/st0 /www /home

Where

Find out what block you are at with mt command:

# mt -f /dev/st0 tell

This does not appear to work on my version of the software. In theory the Status option has a line for block number=, but surprising after completing a backup it seems to still return 0. If I ever work out why this is, I will update this entry.

List

Display list of files on tape drive:

# tar -tzf /dev/st0

Restore

E.g. /www directory

# cd /
# mt -f /dev/st0 rewind
# tar -xzf /dev/st0 www

E.g. /home/test directory

# cd /
# mt -f /dev/st0 rewind
# tar --checkpoint -xvvzpkf /dev/st0 home/test

--checkpoint : provide occasional checkpoint messages
-x : extract
-v : verbosely
-v : even more verbosly
-z : uncompress
-p : retaining permissions
-k : leaving existing files alone
-f /dev/st0 home/test

Unload

Unload the tape:

# mt -f /dev/st0 offline

Status

Display status information about the tape unit:

# mt -f /dev/st0 status

Erase

Erasing the tape may take hours and there is not normally any need to do this; simply rewind the tape before performing backup, or use the mt command to position at the beginning of the tape.

# mt -f /dev/st0 erase

Moving about the tape

You can go BACKWARD or FORWARD on tape with mt command itself

Go to end of data

# mt -f /dev/st0 eod

Goto previous record

# mt -f /dev/st0 bsfm 1

Forward record

# mt -f /dev/st0 fsf 1

Restore

This code has not been checked or tested:

• Check status of tape: mt -f /dev/st0 status
• Go to the directory where you want to restore your file(s).
• Go to the right file on the tape with the following commands:
  • Check file number and position in file: mt -f /dev/st0 status
  • Advance one file: mt -f /dev/st0 fsf 1
  • View contents of tar-file: tar -tvf /dev/st0
  • Go back one file: mt -f /dev/st0 bsf 1

  • If you are in the last block of a file and you should be at the beginning of the file, do the following:

    mt -f /dev/st0 bsf 1

    mt -f /dev/st0 fsf 1

  • And check with: mt -f /dev/st0 status
• Extract your file(s): tar -xvf /dev/st0 ~files~
• Rewind and eject tape: mt -f /dev/st0 offline

Hardware Compression

It may be possible to switch off and on the hardware compression on the:

# mt -f /dev/st0 compression 0
# mt -f /dev/st0 compression 1

Other people report replacing the 0 with “off” and 1 with “on”.

References

• man tar
• man mt
• Howto: Linux Tape Backup with MT and TAR commands
• How to use the DAT-tape with Linux
• Verify tar command tape backup under Linux or UNIX
• Tape drives naming convention under Linux
• Backup home directories in Linux
• Howto: Use tar command through network over ssh session
• Linux Set the Block Size for a SCSI Tape Device

$ dd if=/dev/sda bs=1M | ssh root@blah "cat > /root/disk.img"

Haven’t tried it yet, but it sounds incredible. The bs=1M is essential or the copy will take forever.

# a2enmod ssl
# cd /etc/apache2/
# mkdir ssl
# openssl req -new -x509 -days 3650 -nodes -out /etc/apache2/ssl/apache.pem -keyout /etc/apache2/ssl/apache.pem
# chmod 600 /etc/apache2/ssl/apache.pem
# /etc/init.d/apache2 restart

When generating the certificate it prompts for “Your name”, this should be the name of your site e.g. host.example.com.

SSLEngine on
SSLCertificateFile /etc/apache2/ssl/apache.pem

Step 1 - Install bootchart

# aptitude install bootchart

Step 2 - Update Grub

# nano /boot/grub/menu.lst

Look for a line that looks like this:

# altoptions=(single-user mode) single

And add a similar line after it that looks like this:

# altoptions=(single-user mode) single
# altoptions=(bootchart) init=/sbin/bootchartd

Then update your grub entries by typing:

# update-grub

Step 3 - Reboot

Reboot, making sure that you select the bootchart option from the grub menu.

Once the boot has finished, open a terminal and type:

$ bootchart

Step 4 - Analyse

You chould find a new png image in your current directory (the one you ran the bootchart command from) called bootchart.png. Open it, analyse it and write a really good howto on how to optimise your system, and then publish it on the web!

Good luck!

$ cd /
$ tar cf - opt | (cd /archive; tar xf - )

You cannot run this command as “sudo”, if you need root access for your copy, then you will need to execute a “sudo su” or log in as root.

What the command will do

With any command that you are given by someone, you should always check what that command will do:

$ tar cf - opt | (cd /archive; tar xf - )
c = create
f - = file stdout
opt = source path
| = pipe all above to...
(
cd /archive;
x = extract
f - = file stdin
)

So this command will pipe a new archive from opt to stdout, which it will then recreate in /archive.

This will copy /opt into /archive/opt, preserving permissions, file modification times etc.

Read man tar for more details.

Sometimes you want to catalogue your movies with thumbnail images from the movie.

The Solution

$ ffmpeg -itsoffset -240  -i themovie.mpg -vcodec mjpeg -vframes 1 -an -f rawvideo -s 320x240 thumbnail.jpg
$ ffmpeg -itsoffset -240  -i themovie.mpg -vcodec png -vframes 1 -an -f rawvideo -s 320x240 thumbnail.png

Example Script

The following example script should be saved in /usr/local/bin or somewhere in your path. As you can see, this script will run through all the movies in the current directory and create thumbnails in the default mythtv MythVideo directory.

This script was written to perform a particular task, and a more generic script would be better.

#!/bin/bash
# Creates a thumbnail of an mpeg

for a in *.mpg; do
    if [ -f "$a" ]; then
        b="${a%.mpg}"
        if [ -f "/home/mythtv/.mythtv/MythVideo/${b}.jpg" ]; then
            echo "/home/mythtv/.mythtv/MythVideo/${b}.jpg already exists"
        else
            ffmpeg -itsoffset -240  -i "$a" -vcodec mjpeg -vframes 1 -an -f rawvideo -s 320x240 "/home/mythtv/.mythtv/MythVideo/${b}.jpg"
        fi
    fi
done

Thanks to Google I came across the following command (part of libxml2-utils):

$ xmllint --noout --schema schema.xsd file.xml

I find it amazing that all this time I had the perfect command pre-installed on my Linux desktop and I never knew.

We often use pdftk for command line pdf, but I hadn’t delved deeply into the features. But using pdftk all I had to do was:

$ pdftk A=main.pdf B=2nd.pdf C=3rd.pdf cat A1-24 B A25 C output final.pdf

In other words the final document (final.pdf) is pages 1-24 of document A (main.pdf), the whole of document B (2nd.pdf), page 25 of document A (main.pdf) and the whole of document C (3rd.pdf).

For a command line program, I think that is stunningly intuitive. And best of all it was instantaneous and there was no loss of quality.

pdftk can seemingly do just about anything with pdfs, including encrypt, decrypt, repair, burst and rotate.

pdftk is installable from the Debian repos and typing pdftk --help gives you a handy set of usage examples, so that you don’t have to re-learn it every time you use it.