Archive

Archive for December, 2005

Forest Jump – priority queues at Amazon?

December 16th, 2005 No comments

Earlier today I was dismayed to receive an e-mail from Amazon informing me that the expected delivery date for five Christmas presents I had ordered was 26-30 December. Predictably, this is entirely my own fault through placing the order too late, and expecting delivery from the UK to Norway in good time. I’d already upgraded to Priority Mail service, which made no difference whatsoever, but I had kept the default option of having all my items dispatched together.

I had difficulty obtaining information on when each of the individual items in the order would be in stock, so in a last-ditch attempt to get something out of Amazon before Christmas I selected the option to dispatch each item separately as it became available.

Amazon duly split my original order of five items into one order of four items and a second order with the remaining item. To my astonishment the predicted delivery date for both orders was 20-21 December – some six days earlier and in plenty of time for Jule!

Of course, this defies all logic – unless Amazon is operating some sort of priority queue system – and I have just jumped the queue because my multiple dispatch priority mail order is more profitable to Amazon than my single dispatch priority mail order – which suggests Amazon is making a healthy profit on its shipping charge.

Categories: Business & Economics, Internet Tags:

Gangsta Wrappers

December 5th, 2005 No comments

“Guns don’t kill people. Wrappers do.”

I doubt Welsh satirical rap crew Goldie Lookin Chain were thinking of the same wrappers as me when they wrote the above lyric. Nonetheless, the subject of when to rap and when to wrap is far more contentious amongst developers and their technical managers than I’d ever imagined.

The situation is something like this: Your project needs to use functionality provided by a third-party – most likely in the form of a library to link into your application. You have two choices:

  1. Throw caution to the wind, and use the library API directly
  2. Painstakingly write a wrapper for the library to mitigate against the risks inherent in using code over which you have no control

How do you decide? I believe this is, or at least should be, a business decision. Is it more cost-effective to wrap the third-party library than to not wrap it over the life of the software: Yes or No?

Pros and cons

Wrapper Pros:

  • Insulate against future API changes in the library – useful if the library is likely to be used from many places in our own code
  • Keep open the option of swapping out the library for an alternative in future. This decreases dependence on the library vendor.

Wrapper Cons:

  • The cost of writing, debugging, testing and maintaining the wrapper.
  • The cost of having a non-standard API to the library functionality

The arguments

Hiring: You can probably hire programmers with knowledge of the third-party library in question. There may be a very large, and growing, number of knowledgeable programmers with experience of this particular API in the market and eager for a job. They require no training cost, or months to acclimatise to a unique coding environment. Hiring developers is expensive in time before they become productive – and you’re probably planning on using the new library precisely so you can have programmers being more productive. The prospect of a job working with the API directly may be attractive to potential recruits – it is something they know they can bring to the job. The prospect of working with a proprietary, undocumented, and doubtless buggy wrapper is not.

Documentation: Many third part libraries are excellently documented. Even if the documentation is not perfect it is nearly always superior to in-house API documentation, because this API is their product so it has to be good if they want you to use it. Furthermore, there may be an active mailing list and user community out there providing support in addition to library vendor.

User base: The library may be widely used – probably more widely used than the application you are thinking of integrating it into. Your code may only have tens or hundreds of users. The library code may have many thousands or even millions of users. It goes without saying that the third-party library has already received vastly more real-world testing than your own code will ever receive. Remember, the reason that you are considering it is that it is tried and tested.

Core Business: Is the functionality provided in the third-part library your business? If your core business is database software, should you be spending your time on a user interface toolkit wrapper? Probably not. You should stick to innovating in your own area, and supporting that process with off-the-shelf solutions for the peripheral services you need. You create value for your customers by delivering your core technology. What is the value proposition for the client the paying you to create a user interface toolkit wrapper? Or is this just architecture astronautics?

Russian Dolls: Is the third-party library already a wrapper? Many libraries exist largely to abstract-away implementation differences between platforms – for example threading libraries or user-interface toolkits. If this is a purpose you had in mind for your wrapper, don’t bother. Its been done already.

Russian Dolls
A derivative of GFDL image from Wikipedia article Russian Dolls

Fear and Risk: What are you afraid of? What is your attitude to risk? Wrapping all third-party libraries is a risk-averse strategy. Lets examine some possible scenarios that could cause you future problems:

  1. You find a bug in the library. Either the vendor fixes it in their next release, or if you are fortunate enough to have access to the library source, you fix it yourself.
  2. The library vendor is subject to a hostile takeover in an attempt to ‘embrace and extend’ their success. This is potentially the most damaging scenario – but you need to look carefully at the real risks here. Is a new owner likely to stop supporting the library? If you have access to the source code the risk is reduced. Will the product be withdrawn? Are alternatives available?
  3. A fabulous alternative but currently hypothetical library is released five years from now. You’re worried that at as some indeterminate time in the future you might want to use that instead. No problem: Why not defer writing a wrapper until the new-fangled alternative library is a reality, and then write a wrapper for the new library that makes it API compatible with the existing library? If it never happens, we save the cost of writing and maintaining a wrapper!
  4. The library vendor introduces some radical API changes between major versions: Your invest some time an effort in porting the code – this will probably be less effort than the cumulative effort of maintaining a wrapper. Maybe they even supply tools to help you. Has the library historically had a stable API? If its version 0.9 you should be more concerned than if its version 6.4. Are changes likely to me minor or earth shattering? Probably the former. Is this upgrading effort so much more than maintaining your own wrapper for years? Obviously you should take the nature of the library into account. For example DirectX is continually chasing the breakneck release of new graphics hardware – whereas OpenGL has historically been a stable API

Wrapper Design: How much effort would you be prepared to invest in writing a wrapper? Are you going to hide all the details of the library? To wrap every type returned or passed to it? If you’re using C++ will you use the Cheshire Cat idiom to prevent any types from the library being visible in your wrapper API – even though doing this essentially requires to write both an inner wrapper and out wrapper – a great concept but certainly more effort.

Leaky Abstractions: How ‘thick’ will you wrapper be? Are you planning to add behaviour or present different abstractions to clients of the functionality? If so, be wary of combining a simple wrapper which merely insulates from changes or bugs in the third-party library with a library which layers substantial new behaviour or abstractions. These different tasks may be better separated into different layers. Any abstraction written around a single reference implementation will leak like a sieve.

Multiple Languages: Is your wrapper to provide an existing API in another language? Mixing language can be a powerful technique but beware if your wrapper needs to overcome a significant ‘impedance mismatch’ between the languages in question. Using Java from Jython is easy whereas using C++ from Java through JNI requires much more discipline and understanding.

Conclusion

Ultimately its for you to decide, based on your own circumstances – but in general I would suggest that the effort involved in wrapping third-party libraries rarely pays off.

Categories: architecture, computing, software Tags:

Jython, Swing & Curry

December 3rd, 2005 4 comments

Jython is currently, by far the most useful of the the three main Python implementations, Jython, CPython and IronPython. It allows access to the Java standard library which often provides more standard, better defined, and certainly better documented alternatives to much of what is in the Python standard library. It is also more cross-platform than .NET bound IronPython and offers superior multithreading support to CPython.

Its well known within the Jython community that having first-class functions in Python makes development with Swing more productive in Jython than it is with Java, by allowing us to attach functions or methods directly to Swing callbacks such as actionPerformed, without the need for anonymous classes or excessive need for interface implementation. This example taken from What’s Good About Jython? shows this in action:

import javax.swing as swing

def printMessage(event):
    print "Ouch!"
        
if __name__== "__main__":
    frame=swing.JFrame(title="My Frame", size=(300,300))
    frame.defaultCloseOperation=swing.JFrame.EXIT_ON_CLOSE;
    button=swing.JButton("Push Me!", actionPerformed=printMessage)
    frame.contentPane.add(button)
    frame.visible=1

The most powerful result of learning different programming languages is being able to apply concepts learned in one language, in another language. This type of ‘technology transfer’ drives progress in many science and engineering fields. This cross-fertilisation has always been possible, but is becoming both more feasible and popular thanks to multiparadigm languages such as Python and Ruby, and wider use of the STL and Boost in Standard C++ which encourage and facilitate functional and higher-order programming style. There are even books on the subject of Higher Order Perl.

Although its had many notable ongoing and recent successes (e.g. Emacs, Darcs, Pugs) the functional community is unfortunately widely perceived as academic or impractical. Indeed, Guido van Rossum has recently threatened to remove some functional programming support from Python, bizarrely whilst simultaneously adding higher-order features such as decorators. This is unfortunate because functional programming style and idioms can go a long way to improving software quality in any language – functions without side-effects are far easier to test, debug and reason about.

For example, the technique of currying - fixing a function argument and returning a new function which takes the remaining parameters can be very useful for development with the Swing UI toolkit supplied with Java.

In this example extracted from one of my own projects, where the user interface features two radio buttons, we look at how some higher-order techniques can be used to aid user-interface programming with Swing. In the code below,

fixed_radio = JRadioButton(text="Regular columns of fixed width",
        actionPerformed = self.handleFixedOption)

separated_radio = JRadioButton(text="Fields separated by spaces,
        commas, or other characters",
        actionPerformed = self.handleSeparatedOption)

two bound-methods are attached to the actionPerformed event of each button.

def handleFixedOption(self, event=None):
    self.controller.setOption(self.controller.FIXED)

def handleSeparatedOption(self, event=None):
    self.controller.setOption(self.controller.SEPARATED)

We don’t actually use the event parameter in this case, but it will be passed to the handler function we provide from actionPerformed, so we must accommodate it. These two methods are identical apart from the value passed to setOption internally, so we could factor out the common code and these two functions with a single function and pass the option value FIXED or SEPARATED in as an extra argument,

def handleOption(self, option, event=None):
    self.controller.setOption(option)

However, we now have a difficulty in that we can no longer bind our new function directly to the actionPerformed handler of the buttons, since actionPerformed expects a function which takes single event argument. We know at coding time which option value should be attached to each button – but we don’t know until the code is run, which event will be passed from actionPerformed. We need to bind the option argument before the event argument. The solution to this is currying.

Rather than writing our own currying facility, we can use on off-the-shelf implementation provided by the Xoltar Toolkit which provides some functional programming support in Python. In particular we use the curry function to curry our handleOption function into some new functions that only take take a single event argument.

from functional import curry

fixed_radio = JRadioButton(text="Regular columns of fixed width",
        actionPerformed = curry(self.handleOption,
                self.controller.FIXED))

separated_radio = JRadioButton(text="Fields separated by spaces,
        commas, or other characters",
        actionPerformed = curry(self.handleOption,
                self.controller.SEPARATED))

The curry function takes the function to be curried as its first argument. In this case we pass the bound-method self.optionHandler. The second argument to curry is the value of the first parameter to the function to be curried, which want tobe fixed to this value in the created function, for example, self.controller.FIXED. The result of the curry function is a new function which accepts one less argument than the original function – so in this case we get a new function which only accepts the remaining event parameter, and is therefore equivalent to the handleFixedOption function we started with, but which we have now avoided having to write.

The material advantage with such a simple example is minimal, but when such techniques are used throughout a project the savings in space and complexity can be substantial. The combination of functional concepts from borrowed from languages such a Haskell, facilitated by tools provided by the Xoltar Tookit, developed in dynamically-typed Jython and exploiting one of the better designed user interface APIs in the form of Swing can be concise and powerful.

Categories: computing, Jython, Python, software Tags: