Tag Archives: Linux

The Internet, the Web and an old book

Cover of Running Linux bookNot long ago, I was explaining to a translator the difference between the Internet and the Web. Understandably they thought they were the same thing, as most people do.

Jump forward a few weeks and I’m packing boxes ready to move house, wondering what I can throw out. A dusty edition of Running Linux from 1996 — surely that can go, being so out-of-date? But flicking through it I noticed a chapter devoted to “The World Wide Web and Mail” and this little gem:

The WWW provides a single abstraction for the many kinds of information available from the Internet.

And there you have it. Much more succinct than my long-winded attempt at explaining the difference. But the part that made me smile was this:

The World Wide Web (WWW) is a relative newcomer to the Internet information hierarchy. The WWW project’s goal is to unite the many disparate services available on the Internet into a single, worldwide, multimedia, hypertext space. Although this may seem very abstract to you now, the WWW is best understood by using it.

A page from Running LinuxReading this 17 years after it was written, it almost seems quaint — it’s hard to imagine now that readers of a technical manual would not know what the Web is. And yet because the book assumes no previous knowledge it manages to teach a concept in a way that’s clear and stands the test of time.

Who says technical books lose their value as they get older?

How to convert videos to WebM with FFmpeg/AVConv

After lots of trial and error each time I convert a video to WebM, I finally got around to posting this so I don’t forget next time. In a nutshell, here’s the conversion command that works for me:

avconv -i myvideo.mp4 -acodec libvorbis -aq 5 -ac 2 -qmax 25 -threads 2 myvideo.webm

What is this doing? Let’s go through it bit by bit. Assuming we have a video called myvideo.mp4, the simplest way to convert to WebM is with this little line:

avconv -i myvideo.mp4 myvideo.webm

Easy, but the quality will likely be rubbish hence the use of a few flags. The flags can be divided into three sorts: audio, video and the transcoding itself.

Audio flags

FFmpeg/AVConv outputConcentrating on the audio first, we should specify the audio codec which for WebM is Ogg Vorbis: -acodec libvorbis

In later versions the audio codec is Ogg Vorbis by default but personally I specify it just in case.

The quality can be adjusted with the -aq flag from -1 to 10 with a higher number meaning better quality. I’ve found 4 or 5 to be more than adequate.

The number of channels, e.g. mono (1) and stereo (2), is controlled with the -ac flag.

Video flags

Moving on to the video quality and thankfully it’s nice and simple. Like the audio, we can specify a quality level. With the libvpx library used for WebM, this is actually a quantization level, set with the -qmin and -qmax flags ranging from 0 to 51. In my tests, setting qmin makes no difference so I ignore it. Setting qmax effectively controls the maximum compression that will be applied to each frame. In other words, a higher qmax means higher compression, which results in lower quality and smaller file size. Obviously you should adjust this to whatever’s best for your circumstances, but I’ve found 25 to be a good starting point.

Note that with both the audio and video, setting flags for bitrate (-ab for audio, -b for video) makes little or no difference. Instead, setting quality flags indicates the use of a variable bitrate.

Transcoding flags

Finally, I tend to also use the -threads flag. This simply sets how many CPU threads to use when transcoding. If your PC has multiple cores then a higher number means faster processing but with less spare capacity for running other programs. Incidentally it’s also possible to do 2-pass encoding with WebM using the -pass flag.

FFmpeg naming confusion

Note that due to what seem to be political reasons, using the ffmpeg command in Ubuntu results in a sad message.

Was:

*** THIS PROGRAM IS DEPRECATED ***
This program is only provided for compatibility and will be removed in a future release. Please use avconv instead.

Now:

ffmpeg: command not found

It turns out that FFmpeg is very much alive, as is Libav (avconv), both with similar goals but with slightly different ways of doing it. I don’t follow the details and arguments but for practical purposes, using either the ffmpeg or avconv command is fine for converting videos to WebM with the above flags (at the time of writing). Of course, this may change eventually but for the sake of regular users, I hope not.

Unfortunately whatever FFmpeg/Libav disagreement there was has resulted in ffmpeg being removed from Ubuntu and possibly other Linux distros. For the transcoding commands in this post at least, the parameters are the same so if you have problems using avconv try with ffmpeg and vice versa.

How to wipe a hard drive with Linux

An opened hard driveI have an old hard drive I want to throw away but I don’t want any remaining photos, financial documents or other personal data getting in the wrong hands. In other words, I want to completely and securely wipe the drive. With Linux there are several options and after a bit of research, here’s what I consider the best way.

Firstly, programs such as fdisk, cfdisk or GParted are not sufficient to fully erase data—they just edit a drive’s partition table. What we want is something more thorough. The standard way to do this is to overwrite data on a drive with randomly-generated data. There are several command-line programs to do this (see below for a comparison) but I’ve chosen shred. Its default is three passes (overwriting all data three times) which I’m comfortable with, but you can specify more if you like, e.g. -p 10 for 10 passes.

WARNING! You don’t need me to tell you that accidentally erasing the wrong data could cause you big problems. Please be careful and check, then double-check each command and especially drive name before pressing Enter.

Step 1.
Plug in the hard drive and find its name (sdb, sdc, etc.). Use

dmesg

and look for something like sdb: sdb1

Step 2.
Unmount each partition of the drive (sdb1, sdb2, etc.), e.g.

sudo umount /dev/sdb1

Step 3.
Type the following command for writing random data to the drive three times (default). DON’T PRESS ENTER.

sudo shred -f -v /dev/[your drive name]

Step 4.
Double-check that the drive you’ve specified is the correct one to wipe. OK, now you can press Enter.

Programs for deleting data

badblocks

Purpose: “Search a device for bad blocks”

sudo badblocks -w -t random -p 1 -s /dev/sdb

Summary: A single (-p 1) overwrite (-w) with random data (-t random), showing progress (-s).
Time for 6GB: 16 mins 20 secs

dd

Purpose: “Convert and copy a file”
Notes: You can’t see your progress or specify multiple passes.
Got stuck with faulty drive.

sudo dd if=/dev/urandom of=/dev/sdb

Summary: A single overwrite with random data.
Time for 6GB: 1 hr 5 mins

shred

Purpose: “Overwrite a file to hide its contents, and optionally delete it”

sudo shred -f -v -n 1 /dev/sdb

Summary: Force (-f) a single (-n 1) overwrite with random data, showing progress (-v).
Time for 6GB: 10 mins 6 secs

wipe

Purpose: “Securely erase files from magnetic media”
Notes: Not in default Ubuntu (sudo apt-get install wipe). After running this, I had a few errors running the other commands on the same drive.

sudo wipe -kD -i -q -Q 1 /dev/sdb

Summary: A single (-q -Q 1) overwrite with random data, showing progress (-i), keeping the device’s inode intact (-kD).
Time for 6GB: 9 mins 37 secs

Further information

Smashing a hard disk with a hammerWiping the drive as explained above is good enough for most purposes but forensic experts have amazing skills and tools at their disposal. It may theoretically still be possible to access some of your deleted data. The most secure way to prevent people accessing any data left on your drive is physically drilling, crushing and breaking up the drive, then disposing of the parts in various locations. And encrypting the drive in the first place with TrueCrypt, for example, is also a good idea.

Related links

How to create a DVD-compatible video in PiTiVi

Screenshot of PiTiVi settingsI wanted to edit one of my geeky songs in PiTiVi and burn it to DVD. After searching around and trying a few things it seemed to work, so for reference, here’s how I did it.

Important: Save the PiTiVi project before you start, as I’ve found it to be prone to crashing.

  1. Once the video and audio are ready, click “Render project”
  2. Click “Choose File” and enter a filename with the extension .mpg
  3. Click “Modify” and select either 576p PAL or 480p NTSC depending on your region
  4. Change the export container to “FFmpeg MPEG-2 PS format (DVD VOB)”
  5. Change the audio codec to “FFmpeg ATSC A/52A (AC-3)”
  6. In the audio settings, change the Bit Rate to 256000
  7. The video codec should be fixed at “FFmpeg MPEG-2″
  8. In the video settings, change the Bit Rate to 6548000
  9. Click “OK”, then “Render”

You can now use a program such as Bombono to burn straight to DVD, or DeVeDe (slightly better, in my opinion) to create an ISO which you can burn to DVD with Brasero or other tool.

Incidentally, before rendering I used “noise removal” and “normalize” in Audacity to create a clean soundtrack. I imported this into PiTiVi and removed the original audio. Result: success!

How to show Japanese text in Evince

A quick tip that might help others (or me, the next time I forget)…

I found that Evince, Ubuntu’s default PDF viewer, doesn’t display Japanese characters, at least on my non-Japanese system. After a quick search it seems the answer is nice and simple – install the poppler-data package.

So, either use the Synaptic Package Manager or the following terminal command:

sudo apt-get install poppler-data

And that’s it!

This should work for displaying Chinese and Korean characters as well.

Opera update: “Requires installation of untrusted packages”

On two separate installations of Ubuntu 10.10 I got the following message when trying to update Opera using Ubuntu’s Update Manager:
“Requires installation of untrusted packages”

Unfortunately, the only option available was to click the Close button with no help to fix the problem. Fortunately, there’s a solution, also on My Opera, that worked for me: my.opera.com/vinhphuc121583/blog/fix-error-requires-installation-of-untrusted-packages

A couple of points to note:

  • You should add the Opera GPG key, as explained here: help.ubuntu.com/community/OperaBrowser
  • The final
    sudo apt-get update

    doesn’t update Opera, but using the Update Manager again works with no error message.

Thank you Bùi Vĩnh Phúc!

UPDATE: Things seem to work again now so instead of following the instructions in the linked blog post, you should just be able to add the Opera GPG key (even if you’ve done it before) and update as explained above. Official information is here:
my.opera.com/desktopteam/blog/2011/01/25/an-update-reminder-to-ubuntu-linux-mint-and-debian-users-or-anyone-else-on-a-d

How to create Ogg videos in Linux (with GUI)

Following up on my previous post about converting videos to Ogg Theora, there’s also an easy way for those who like a GUI.

Screenshot of the OggConvert programAlthough it’s possible to use VLC for Ogg encoding, I had problems with the audio and video being out of sync. Fortunately I found another way – enter OggConvert.

Written in Python by Tristan Brindle, there are packages for the major Linux distributions as well as a version for Windows. Not only that, it’s open source (GNU LGPL), preserves metadata when converting and is extremely easy to use:

  1. Fire up OggConvert
  2. Select the source video (or audio) file
  3. Click convert
  4. Voila! Enjoy your Ogg video

By default, the target file is saved in the same directory and with the same name as the source file but with an .ogg or .ogv extension. The default quality settings seem a bit low but should be adequate for most purposes.

When done, your Ogg video is now ready to use in your HTML5 pages:

<video src="video.ogg" width="320" autobuffer controls>
  <p>Sorry, your browser can't play this video but you can
  <a href="video.ogg">download it here</a>.</p>
</video>

How to create Ogg videos in Linux

Ogg Theora is the codec most supported by modern HTML5-capable browsers, so how can you convert your videos to it? Pretty easily, it seems.

There’s a cross-platform command-line application called ffmpeg2theora which does a good job of converting videos to Ogg Theora. Usage couldn’t be simpler:

ffmpeg2theora input.mpeg

If the audio is out of sync with the video, either try adding the –sync option or convert it to MPEG format using ffmpeg first, and then use ffmpeg2theora.

For those all-important tags (except ‘comment’) you can use parameters like so:

ffmpeg2theora input.mpeg --title "My spiffing video" --artist "Daniel" --license "http://creativecommons.org/licenses/by-nc-sa/3.0" --date "2010" --organization "Opera Software" --location "Tokyo, Japan" 

Or alternatively there’s oggz, available in the oggz-tools package in Ubuntu:

oggz-comment input.ogv TITLE="My spiffing video" ARTIST="Daniel" LICENSE="http://creativecommons.org/licenses/by-nc-sa/3.0" DATE="2010" ORGANIZATION="Opera Software" LOCATION="Tokyo, Japan" COMMENT="Best viewed on a screen" -o output.ogv

To embed your converted video in an HTML5 page, use something like this:

<video src="video.ogv" autobuffer controls>
    <p>Your browser can't display the open Ogg Theora codec but you can download it from here: <a href="video.ogv">Video</a></p>
</video>

N.B. For Mac, there’s Exom which has ‘HTML5′ as one of its output options (thanks Bruce).

UPDATE: If you’d prefer to not use the command line, there’s a GUI alternative for easy Ogg conversion.

How to install Truecrypt on Fedora

I’ve tried on several versions of Fedora but never really been able to install Truecrypt. Either it’s resulted in lots of errors or I’ve given up after getting tired of fruitless searching and reading.

At last I’ve found a guide that’s not too long and works!

It’s written for Truecrypt 6.2 on Fedora 11 but worked perfectly for me with Truecrypt 6.3a on Fedora 12:

penguinenclave.blogspot.com/2009/07/truecrypt-62-install-guide-for-fedora.html

Of course, it would be best if Truecrypt was available in Linux repositories but from what I understand, the way it’s written makes it very difficult for distros to package and maintain. Even so, the greatness of Truecrypt still makes it worth doing yourself.

How to watch videos on a Nokia smartphone

The Nokia N96 smartphoneMy new Nokia N96 (yes, and then they immediately released the N97!) may not be as intuitive as the iPhone but is packed with far more features and is so much more open to exploration that it’s a keeper. Not only that but clever @LonelyBob gave me some help last night to fix a problem I’d been battling with – video conversion.

Like him, I tried MEncoder which (unusually) didn’t do the trick and then gave FFmpeg a go. As usual, it requires a load of options and arguments that I don’t fully understand but converts almost any format into any other format – in this case an MP4 file. Trouble was, everything I tried resulted in a playback error on the phone. After exhaustive searching it seems the MP4 needs to be in a mobile-friendly wrapper for which I used MP4Box, found in the “gpac” package on Linux. That worked nicely but the extra step is a hassle. Enter @LonelyBob and his magic one-liner. I edited it a bit for my version of FFmpeg (3:0.svn20090303-1) and this is what’s left:

ffmpeg -y -i input.mkv -vol 256 -acodec libfaac -ac 2 -ar 44100 -ab 64000 -f 3gp -vcodec mpeg4 -s 320x240 -r 25 -b 256000 output.mp4

Of course, input.mkv can be almost any type of video file. The resulting output file is small but not great quality so you could try changing the bitrates, etc. to something like:

  • -ab (audio bitrate): 96000
  • -b (video bitrate): 384000
  • -s (dimensions): 320×176 (for widescreen)

N.B. If you get an “Unknown encoder: libfaac” message, try using “-acodec aac” instead.

Oh, and it’s worth the effort – the N96′s screen is fantastic.