Archive for the ‘Programming’ Category

« Older Entries

C++ Exception Safety Guarantee

Wednesday, January 14th, 2009

After stumbling on this one in a recent telephone interview, I thought I’d refresh my memory.

The following from Anthony Williams’s ACCU Overload Journal article last August provides a succinct summary of the topic.

The Abrahams Exception Safety Guarantee

These guarantees were first documented by Dave Abrahams when the C++ Standards committee were working on the 1998 C++ Standard. The idea is that code should provide one of the three guarantees – if it doesn’t, then an exception occuring in your code will result in leaked resources or corrupt data structures or both. The guarantees are:

The no-fail (or no-throw) guarantee

This is the strongest of all guarantees. A function that provides this guarantee will not throw any exceptions, and will not fail. All destructors should provide this guarantee, as should important operations like swap which provide the building blocks for the code that uses them to provide suitable exception safety guarantees.

The strong guarantee

A function that provides this guarantee is all or nothing: if it fails, then any effects are rolled back so the state of the data structure is the same as it was on entry. This requires that the function doesn’t do anything irreversible (like perform I/O), and that there are suitable operations that provide the no-fail guarantee which can be used to commit or roll back the changes.

The basic guarantee

This is the basic level you should strive for in all code: if a function fails, then it must leave the data structures in a valid state, even if that state differs from the original. For example, failure to insert a new item into a container must leave the container in a valid state, even if all the existing items have been deleted.

Any code that doesn’t provide even the basic guarantee is not exception safe.

Exceptions Make for Elegant Code, ACCU Overload Journal #86, August 2008

Writing exception safe code is hard. If you have any doubts about that statement consider the following challenge from Herb Sutter: Guru of the Week 8. In Exceptional C++, Sutter expands on his earlier post and provides the following guidelines –

Observe the canonical exception-safety rules: (1) Never allow an exception to escape from a destructor or from an overloaded operator delete() or operator delete[](); write every destructor and deallocation function as though it had an exception specification of “throw()“. (2) Always use the “resource acquisition is initialization” idiom to isolate resource ownership and management. (3) In each function, take all the code that might emit an exception and do all that work safely off to the side. Only then, when you know that the real work has succeeded, should you modify the program state (and clean up) using only nonthrowing operations.

The standard for the C++ Standard Library contains the guarantee that no destructor operation defined in the library itself will throw an exception. The C++ Standard, however, does not enforce this requirement for all C++ code so your compiler will not prevent you from creating a class that breaks the golden rule: never throw exceptions from a destructor.

Anyone looking for more information should consult the links above and also look at Abrahams’s essay: Exception-Safety in Generic Components: Lessons Learned from Specifying Exception-Safety for the C++ Standard Library.

Posted in Programming | 2 Comments »

Retrieving Rapidshare Files with Python

Friday, January 2nd, 2009

A cursory google search will reveal several scripts for retrieving rapidshare files using python, but each one I’ve seen delegates the actual retrieval to wget.

This is not necessary.

Rapidshare uses basic authentication to identify logged in members and urllib2 can handle this easily.

The following method would do the trick without the need to call external executables:

def rapidget(url. login, password):
    "Retrieve files from rapidshare using only python"
    request = urllib2.Request(url)
    base64string = base64.encodestring('%s:%s' % (login, password))[:-1]
    request.add_header("Authorization", "Basic %s" % base64string)
    i = url.rfind('/')
    filename = url[i+1:]
    print url, "->", filename
    file = open(filename, 'wb')
    handle = urllib2.urlopen(request)
    buffer = ''
    buffersize = 1024*1024
    while True:
        buffer = handle.read(buffersize)
        if not buffer:
            handle.close()
            file.close()
            break
        file.write(buffer)
        buffer = ''
        print '.',

This assumes, of course, that you have an account at rapidshare.

Posted in Internet, Programming, Uncategorized, howto, python | Comments Off

Deleting Messages from an IMAP Folder Using Python

Saturday, August 16th, 2008

I was asked to help delete 16,980 messages from an IMAP spam folder the other day. No email client could handle it without crashing. Even mutt choked after several hours of valiantly struggling.

Python to the rescue. Rather than write a script to do this I ran each command from the python shell. It’s a very addictive way of working because you get instant feedback.

import getpass, imaplib
M = imaplib.IMAP4_SSL("imap.gmail.com")
M.login("yourusername@gmail.com", getpass.getpass())

Now we’re in. Let’s see what directories exist.

M.list()

Pick the offending directory.

M.select("[Gmail]/Spam")

View the messages.

typ, data = M.search(None, 'ALL')
for num in data[0].split():
....typ, data = M.fetch(num, '(RFC822)')
....print 'Message %s\n%s\n' % (num, data[0][1])

Now delete them all and close the connection to the mailserver. To delete a message in IMAP, you need to set the delete flag on it then expunge the folder.

typ, data = M.search(None, 'ALL')
for num in data[0].split():
....M.store(num, '+FLAGS', '\\Deleted')
M.expunge()
M.close()
M.logout()

(To be fair to google I must point out that gmail was not the offending mailserver although I’ve used them in the sample code above.)

Posted in Internet, Programming, howto, python | 5 Comments »

Patching K3B on Kubuntu Hardy Heron

Tuesday, May 6th, 2008

I’ve recently built a new desktop machine – the subject of another post when I get time – and because the hardware is fairly recent I decided to try Ubuntu which I knew would support it rather than switch to an unstable branch of Debian.

My first impressions of Hardy Heron – Ubuntu’s latest release; I’m using the variant with the KDE desktop – are very positive. The installer is excellent and the hardware detection very impressive. As a comparison, it took me the better part of a day to install a fairly vanilla Windows XP Pro SP2 on the first partition of the box. Nearly every driver install required a reboot as did many of the security updates. By contrast, it took less than an hour to download and install Kubuntu and the installation includes a far greater range of packages than I have on the windows partition.

The one fly in the ointment has been a problem with K3b. If I started with a blank CD in the drive, all was fine but launching it with either an empty drive or a blank DVD caused the process to hang for a long while and finally launch with the warning:

“No CD/DVD writer found.
K3b did not find an optical writing device in your system. Thus, you will not be able to burn CDs or DVDs. However, you can still use other K3b features like audio track extraction or audio transcoding or ISO9660 image creation.”

After this the DVD drive was disabled, making it impossible to even eject until the machine had been shutdown completely and restarted.

Both growisofs and wodim worked from the command line so the problem had to be in the KDE application. Launching K3B from the console reveals debug information and showed the hang up was occurring after the step checking for RAW_R96R:

...
(K3bDevice::Device) /dev/scd0: checking for RAW_R96R

After a long pause the process would continue:

(K3bDevice::ScsiCommand) failed:
                           command:    GET PERFORMANCE (ac)
                           errorcode:  72
                           sense key:  NO SENSE (2)
                           asc:        0
                           ascq:       3
(K3bDevice::Device) /dev/scd0: GET PERFORMANCE length det failed.
(K3bDevice::ScsiCommand) failed:
                           command:    MODE SENSE (5a)
                           errorcode:  0
                           sense key:  NO SENSE (2)
                           asc:        0
                           ascq:       0
(K3bDevice::Device) /dev/scd0: MODE SENSE length det failed.
(K3bDevice::ScsiCommand) failed:
                           command:    MODE SENSE (5a)
                           errorcode:  0
                           sense key:  NO SENSE (2)
                           asc:        0
                           ascq:       0
(K3bDevice::Device) /dev/scd0: MODE SENSE with real length 65535 failed.
(K3bDevice::Device) could not open device /dev/scd0 for reading
                    (No such device or address)
Error while retrieving capabilities.
Could not initialize device /dev/scd0
Could not resolve /dev/scd0
/dev/scd0 resolved to /dev/scd0
(K3bDevice::Device) could not open device /dev/scd0 for reading
                    (No such file or directory)
could not open device /dev/scd0 (No such file or directory)
Devices:
------------------------------
kdecore (KAction): WARNING: KActionCollection::operator+=(): function is severely deprecated.
removing udi /org/freedesktop/Hal/devices/pci_8086_2922_scsi_host_0_scsi_device_                                                                                                                                                 lun0_scsi_generic
removing udi /org/freedesktop/Hal/devices/pci_8086_2922_scsi_host_0_scsi_device_                                                                                                                                                 lun0
removing udi /org/freedesktop/Hal/devices/pci_8086_2922_scsi_host_0
removing udi /org/freedesktop/Hal/devices/storage_model_DVDRW_LH_20A1S
Unmapping udi /org/freedesktop/Hal/devices/storage_model_DVDRW_LH_20A1S from dev                                                                                                                                                 ice /dev/scd0

dmesg reveals the following occuring

May  1 23:52:15 mingus kernel: [ 3580.143800]          cdb ac 00 00 00 00 00 00 00  00 01 03 00 00 00 00 00
May  1 23:52:15 mingus kernel: [ 3580.143801]          res 40/00:03:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
May  1 23:52:15 mingus kernel: [ 3580.455532] ata6: soft resetting link
May  1 23:52:15 mingus kernel: [ 3580.619411] ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
May  1 23:52:45 mingus kernel: [ 3610.751976] ata6.00: qc timeout (cmd 0xa1)
May  1 23:52:45 mingus kernel: [ 3610.751985] ata6.00: failed to IDENTIFY (I/O error, err_mask=0x5)
May  1 23:52:45 mingus kernel: [ 3610.751993] ata6: failed to recover some devices, retrying in 5 secs
May  1 23:52:50 mingus kernel: [ 3615.752086] ata6: hard resetting link
May  1 23:52:56 mingus kernel: [ 3621.263794] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:53:00 mingus kernel: [ 3625.752312] ata6: hard resetting link
May  1 23:53:06 mingus kernel: [ 3631.268013] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:53:11 mingus kernel: [ 3635.804490] ata6: hard resetting link
May  1 23:53:16 mingus kernel: [ 3641.316197] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:53:46 mingus kernel: [ 3670.813264] ata6: limiting SATA link speed to 1.5 Gbps
May  1 23:53:46 mingus kernel: [ 3670.813266] ata6: hard resetting link
May  1 23:53:51 mingus kernel: [ 3675.821377] ata6.00: disabled
May  1 23:53:51 mingus kernel: [ 3675.821404] ata6: hard resetting link
May  1 23:53:57 mingus kernel: [ 3681.736761] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:54:01 mingus kernel: [ 3685.825584] ata6: hard resetting link
May  1 23:52:15 mingus kernel: [ 3580.143800]          cdb ac 00 00 00 00 00 00 00  00 01 03 00 00 00 00 00
May  1 23:52:15 mingus kernel: [ 3580.143801]          res 40/00:03:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
May  1 23:52:15 mingus kernel: [ 3580.455532] ata6: soft resetting link
May  1 23:52:15 mingus kernel: [ 3580.619411] ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
May  1 23:52:45 mingus kernel: [ 3610.751976] ata6.00: qc timeout (cmd 0xa1)
May  1 23:52:45 mingus kernel: [ 3610.751985] ata6.00: failed to IDENTIFY (I/O error, err_mask=0x5)
May  1 23:52:45 mingus kernel: [ 3610.751993] ata6: failed to recover some devices, retrying in 5 secs
May  1 23:52:50 mingus kernel: [ 3615.752086] ata6: hard resetting link
May  1 23:52:56 mingus kernel: [ 3621.263794] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:53:00 mingus kernel: [ 3625.752312] ata6: hard resetting link
May  1 23:53:06 mingus kernel: [ 3631.268013] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:53:11 mingus kernel: [ 3635.804490] ata6: hard resetting link
May  1 23:53:16 mingus kernel: [ 3641.316197] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:53:46 mingus kernel: [ 3670.813264] ata6: limiting SATA link speed to 1.5 Gbps
May  1 23:53:46 mingus kernel: [ 3670.813266] ata6: hard resetting link
May  1 23:53:51 mingus kernel: [ 3675.821377] ata6.00: disabled
May  1 23:53:51 mingus kernel: [ 3675.821404] ata6: hard resetting link
May  1 23:53:57 mingus kernel: [ 3681.736761] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:54:01 mingus kernel: [ 3685.825584] ata6: hard resetting link
May  1 23:54:07 mingus kernel: [ 3691.752971] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:54:11 mingus kernel: [ 3695.841797] ata6: hard resetting link
May  1 23:54:17 mingus kernel: [ 3701.757192] ata6: port is slow to respond, please be patient (Status 0x80)
May  1 23:54:46 mingus kernel: [ 3730.858562] ata6: limiting SATA link speed to 1.5 Gbps
May  1 23:54:46 mingus kernel: [ 3730.858565] ata6: hard resetting link
May  1 23:54:51 mingus kernel: [ 3735.884188] ata6: EH complete
May  1 23:54:51 mingus kernel: [ 3735.884287] ata6.00: detaching (SCSI 5:0:0:0)

It looks like the bug is probably in the libsata code failing to handle an error rather than K3B but I haven’t had a chance to investigate further.

As a quick workaround, download the k3b source and hack bool K3bDevice::Device::getSupportedWriteSpeedsViaGP to return false immediately and shortcircuit the call to ioctl that locks the device. The code will fall back to setting a maximum speed determined by the previous call to getSupportedWriteSpeedsVia2A.

Posted in Linux, Programming, howto | 3 Comments »

Podcasts for Software Engineers (and the Perfect Player: the Nokia N95)

Sunday, April 27th, 2008

I’ve recently discovered an excellent new podcast on software engineering – Software Engineering Radio. I’ve long been a fan of IT Conversations and In Our Time and this is a very welcome addition to my listening schedule.

As I client I’ve been using my Nokia N95 after upgrading the firmware to v20.0.015. The upgraded phone has no difficulty addressing memory cards larger than 4G, vastly improved battery life while playing mp3s, no longer crashes from running out of memory (thanks to on demand paging which also cuts the boot time significantly) and has a podcast client built in that remembers where I left off a particular episode. This is ideal if, like me, you use podcasts to claw back otherwise wasted time and so often digest episodes in several minute chunks.

Oh, and you can sync up your subscriptions and download new episodes direct to the player using wireless which makes it extremely convenient.

I’m working through the Software Engineering Radio backlist at the moment and have particularly enjoyed the following episodes on game development, Erlang, Lisp and an interview with Erich Gamma. You know something must be good when you find yourself looking forward to doing the washing up because it will give you another ten minutes to listen to more of the latest episode. For anyone interested in software development, I’d heartily recommend it.

Posted in General IT, Nokia, Programming, Symbian | 2 Comments »

Amusing Compiler Warning

Saturday, October 27th, 2007

In file included from vhook/imlib2.c:48:
/home/tim/src/ffmpeg/libavformat/framehook.h:25:2: warning: #warning VHOOK is deprecated. Please help porting libmpcodecs or a better filter system to FFmpeg instead of wasting your time writing new filters for this crappy one.

Posted in Programming | Comments Off

The Promise, the Limits, the Beauty of Software

Sunday, January 28th, 2007

Starting with Bjarne Stroustrup’s observation “our civilization runs on software”, Grady Booch offered a thought provoking overview of the history and promise of software at this year’s BCS Turing lecture, taking listeners from the austere beauty of Alan Turing’s 1930s thought experiments through to “the rise of the machines” in 2030.

Booch is an interesting, relaxed and witty speaker, whose asides on the superiority of OS X to Windows, George Bush and Google (“Am I the only one who thinks there’s a company in desperate need of some adult supervision?”) provided comic relief in an at times informationally dense speech.

One point that intrigued me was his observation that much of the history of computing is unrecorded, existing only in the “tribal memory” of the greybeards. He foresees the emergence of both software artists and historians who might translate and record some of the strange beauty of code for non-programmers as well as formally archiving a form of communication in danger of vanishing with the death of its authors.

The full lecture is available as a recording from the link below and is well worth watching.



Posted in Culture, General IT, Internet, Programming | Comments Off

Bjarne Stroustrup on the Problem with Programming

Monday, December 11th, 2006

A few extracts from a great two-part interview with Bjarne Stroustrup: Technology Review: The Problem with Programming,
More Trouble with Programming.

On over-abstraction:

Software developers have neutralized the astounding performance of modern computer hardware by adding layer upon layer of overelaborate [software] abstractions. We seem to have hit the limits of linear speedup for hardware, but in many cases, we could win a couple of orders of magnitude back from the software.

On “the next big thing”:

I don’t know what the next major conceptual shift will be, but I bet that it will somehow be related to the management of concurrency. As programmers, we have been notoriously bad at thinking about lots of things happening simultaneously, and soon our everyday computers will have 32 cores.

On .NET and the JRE:

I want my software to run everywhere it makes sense to run it. I also want to be able to change suppliers of parts of my system if the suppliers are not the best. Obviously, suppliers of huge integrated systems, such as .Net and Java, see things differently. Their claim is that what they provide is worth more to users than independence. Sometimes they are right, and of course some degree of integration is necessary: you cannot write a complete application of any realistic size without introducing some system dependencies. The question is how deeply integrated into the application those system dependencies are. I prefer the application to be designed conceptually in isolation from the underlying system, with an explicitly defined interface to “the outer world,” and then integrated through a thin layer of interface code.

Well worth the read.

Posted in Programming | Comments Off

Debugging the Sick

Wednesday, September 20th, 2006

My father told me an anecdote about a patient who, while clearly very ill, showed a baffling lack of symptoms. To the first doctor he claimed everything was perfectly normal. To the second, he claimed that he’d noticed no change to his normal daily routine. It wasn’t until a the third doctor to see him asked him to step through a “normal” day that a missing key symptom became apparent: “The alarm goes off at 7.30, I get out of bed, go to the bathroom, throw up…”

This isn’t meant as an argument for using debuggers when coding but rather a warning always to challenge assumptions and to fix problems when they occur. Routine blinds us to everything. This can be a blessing in day-to-day living but it’s a curse when problems cascade and the root cause is obscured by habit.

Posted in Programming | Comments Off

Eclipse Quick Fix != Quick Assist

Sunday, August 13th, 2006

If there’s an error in your java code, Eclipse sticks a marker in the gutter with a light-bulb icon. Clicking on the marker or pressing Ctl+1 with the cursor in the offending line brings up a helpful pop-up of suggested fixes for the problem.

I’ve always assumed that this was called Quick Fix. If you right click on the marker, the option is labelled Quick Fix. However, after struggling for a long while to understand why some Quick Fix code I wrote wasn’t showing up in this menu, I’ve come to the conclusion that this is actually Quick Assist and that the label is wrong (I’ve flagged this as a bug).

To launch Quick Fix you need instead to right click on a Warning/Error in the Problems View. Quick Fix and Quick Assist are in fact two different things with confusingly similar names and functionality.

Internally appropriate Quick Fixes are calculated by a class implementing IMarkerResolutionGenerator that extends org.eclipse.ui.ide.markerResolution. The fixes themselves are contained in classes implementing either IMarkerResolution or IMarkerResolution2. By contrast, Quick Assists are calculated by a class extending org.eclipse.jdt.ui.quickAssistProcessors and implement IQuickAssistProcessor. The classes containing the assists implement IJavaCompletionProposal.

So, if you’re developing a plug-in and cannot work out why your Quick Fixes are not showing up, take a closer look. Maybe like me you’re checking in the wrong place and what you meant to write was actually a Quick Assist.

Posted in Programming | 6 Comments »

« Older Entries