September 2005

AI Newsletter

• 21 min read

Introduction

Welcome to our September issue. The main feature this month continues August's AI-in-Python theme with a look at Python for robotics and the Pyro robot-control software. We also have a selection of quotes, and some computer-generated humour. As ever, comments and suggestions are welcome.

An Arc Through AI Space

I came across a few of the quotes below while looking up references for another article. It's an article that hasn't yet worked out, but I thought it would be fun to use them to trace a path through the past - and perhaps future - development of AI. So here goes:

Many smart people have been thinking about the AI problem for a long time. There have been many ideas that have been pursued by sophisticated research teams which turned out to be dead ends. This includes all of the obvious ideas. Most grand solutions proposed have been seen before (about 70% seem to be recapitulations of Minsky proposals)."

"There has been a long-standing opposition within AI between 'neats' and 'scruffies' (I think the terms were first invented in the late 70s by Roger Schank and/or Bob Abelson at Yale University). The neats regard it as a disgrace that many AI programs are complex, ill-structured, and so hard to understand that it is not possible to explain or predict their behaviour, let alone prove that they do what they are intended to do. John McCarthy in a televised debate in 1972 once complained about the 'Look Ma no hands!' approach."

"Conrad Barski from Minneapolis sent me an action shot of the John McCarthy Lisp t-shirt. He writes: '... and since the portrait of John McCarthy is so uncanny, there was no need to explain the shirt to anyone in the audience.'"

  • John McCarthy Lisp T-shirt blog entry at Lispmeister, 2004.

"Lisp has jokingly been called 'the most intelligent way to misuse a computer'. I think that description is a great compliment because it transmits the full flavor of liberation: it has assisted a number of our most gifted fellow humans in thinking previously impossible thoughts."

  • www.paulgraham.com/quotes.html
  • Edsger Dijkstra in his Humble Programmer essay for CACM, 1972. Quoted in Paul Graham's Lisp Quotes.

"Elegance is unnatural, only achieveable at great expense. If you just do something, it won't be elegant, but if you do it and then see what might be more elegant, and do it again, you might, after an unknown number of iterations, get something that is very elegant."

  • Lisp programmer Erik Naggum.

"The language God would have used to implement the Universe."

  • ALU
  • Svein Ove Aas, quoted at The Road to Lisp Survey Highlight Film. This is a compilation of replies to The Road to Lisp Survey, a newbie-by-newbie survey of what led folks to give Lisp a serious try and what they think of it.

"It feels like lightning between your fingertips."

  • Glenn Ehrlich, The Road to Lisp Survey Highlight Film.

"((What ((is) with (all)) of (the) ()s?) Hmmm?)"

"As the release of AutoCAD 2.1 loomed closer, we were somewhat diffident about unleashing Lisp as our application language. This was at the very peak of the hype-train about expert systems, artificial intelligence, and Lisp machines, and while we didn't mind the free publicity we'd gain from the choice of Lisp, we were afraid that what was, in fact, a very simple macro language embedded within AutoCAD would be perceived as requiring arcane and specialised knowledge and thus frighten off the very application developers for whom we implemented it. In fact, when we first shipped AutoCAD 2.1, we didn't use the word 'Lisp' at all - we called it the 'variables and expressions feature'. Only in release 2.18, in which we provided the full functional and iterative capabilities of Lisp, did we introduce the term 'AutoLisp'."

"'AI winter' is the term first used in 1988 to describe the unfortunate commercial fate of AI. From the late 1970ā€™s and until the mid-1980ā€™s, artificial intelligence was an important part of the computer business - many companies were started with the then-abundant venture capital available for high-tech start-ups. By 1988 it became clear to business analysts that AI would not experience meteoric growth, and there was a backlash against AI and, with it, Lisp as a commercial concern. AI companies started to have substantial financial difficulties, and so did the Lisp companies."

  • www.dreamsongs.com
  • From The Evolution of Lisp by Guy Steele and Richard Gabriel.

"The scruffies regard messy complexity as inevitable in intelligent systems and point to the failure so far of all attempts to find workable clear and general mechanisms, or mathematical solutions to any important AI problems. There are nice ideas in the General Problem Solver, logical theorem provers, and suchlike but when confronted with non-toy problems they normally get bogged down in combinatorial explosions. Messy complexity, according to scruffies, lies in the nature of problem domains (e.g. our physical environment) and only by using large numbers of ad-hoc special-purpose rules or heuristics, and specially tailored representational devices can problems be solved in a reasonable time."

  • Must Intelligent Systems Be Scruffy?

"In rule-based, or expert systems, the programmer enters a large number of rules. The problem here is that you cannot anticipate every possible input. It is extremely tricky to be sure you have rules that will cover everything. Thus these systems often break down when some problems are presented; they are very 'brittle'. Connectionists use learning rules in big networks of simple components - loosely inspired by nerves in a brain. Connectionists take pride in not understanding how a network solves a problem."

  • https://www.aaai.org
    Marvin Minsky, from Scientist on the Set: An Interview with Marvin Minsky, in Hal's Legacy, edited by David Stork, 1996. Quoted on the AAAI Reasoning page.

"Despite all the progress in neural networks the technology is still brittle and sometimes difficult to apply."

  • Statistical methods for construction of neural networks, a review of methods for building robust neural nets by Wlodzislaw Duch and Rafal Adamczak, 1998.

"It would be best to start with ready software packages. I recommend our own ones, because they are error-free and involve all our know-how; on the contrary, many commercial packages are of no use."

  • Teuvo Kohonen, replying to the question "What tips would you give to programmers wanting to create self-organizing neural networks?" in an interview with generation5, 2000.

"All too soon, however, the hopes kindled by AI's second age dimmed as well. Using chips and computer programs, scientists built artificial neural nets that mimicked the information-processing techniques of the brain. Some of these networks could learn to recognise patterns, like words and faces. But the goal of a broader, more comprehensive intelligence remained far out of reach. And so dawned the third age of AI. Its boosters abandoned hopes of designing the information-processing protocols of intelligence, and tried to evolve them instead. No one wrote the program which controls the walking of Aibo, a $1,500 robotic dog made by Sony. Aibo's genetic algorithms were grown - evolved through many generations of ancestral code in a Sony laboratory."

"GAs are a terrific approach to searching large, ill-defined spaces, in this case the space of 'nice' melodic ideas. There is also an analogy to the 'population' of licks that most jazz players have in their heads. These licks come and go over time in a manner similar to evolution; ideas that were cool in the past become overused or cliched, so I stop playing them."

  • John Al Biles in a 1998 interview with generation5 about his work on the GenJam Genetic Jammer interactive jazz improviser, probably the only evolutionary computation system that is also a working musician.

"Dealing with ES is sometimes seen as 'strong tobacco', for it takes a decent amount of probability theory and applied STATISTICS to understand the inner workings of an ES, while it navigates through the hyperspace of the usually n-dimensional problem space, by throwing hyperellipses into the deep..."

  • http://www.faqs.org/faqs/ai-faq/genetic/part2/
  • From an account of the Technical University of Berlin's work on Evolution Strategies, one of many detailed descriptions on evolutionary algorithms in part 2/6 of the comp.ai.genetic FAQ by Joerg Heitkoetter and David Beasley, 1993-2001.

"It is raining instructions out there; it's raining programs; it's raining tree-growing, fluff-spreading, algorithms. That is not a metaphor, it is the plain truth. It couldn't be any plainer if it were raining floppy discs."

"My optimism about the future of intelligent machines is based partly on the evolutionary record. Nature holds the patents on high intelligence. It invented it not once, but several times, as if to demonstrate how easy it was. ... The vertebrate retina has been studied extensively. Its 20 million neurons take signals from a million light sensors and combine them in a series of simple operations to detect things like edges, curvature and motion. Then image thus processed goes on to the much bigger visual cortex in the brain. Assuming the visual cortex does as much computing for its size as the retina, we can estimate the total capability of the system. The optic nerve has a million signal carrying fibers and the optical cortex is a thousand times deeper than the neurons which do a basic retinal operation. The eye can process ten images a second, so the cortex handles the equivalent of 10,000 simple retinal operations a second, or 3 million an hour."

It sometimes seems to me that the brain is actually a very shitty computer. So why would you want to build a computer out of slimy, wet, broken, slow, hungry, tired neurons? I chose computer science over medical school because I don't have the stomach for those icky, bloody body parts. I prefer my technology clean and dry, thank you. ... The brain has to sleep, needs food, thinks about sex all the time. Useless! I always say, if I wanted to build a computer from scratch, the very last material I would choose to work with is meat. I'll take transistors over meat any day. Human intelligence may even be a poor kludge of the intelligence algorithm on an organ that is basically a glorified animal eyeball."

"I claim that the soul, spirit, or consciousness may exist, but for most people, most of the time, it is almost infinitesimally small, compared with the robotic machinery responsible for most of our thought and action. ... That's not to say that some people can't be more enlightened than others. But for the vast herd out there, on average, consciousness is simply not a significant factor. Not even a second- or third-order effect. Consciousness is marginal. I say this with such confidence because of my experience building robot brains over the past seven years. Almost everything people ever say to our robot falls into one of about 45,000 categories. Considering the astronomical number of things people could say, if every sentence was an original line of poetry, 45,000 is a very, very small number."

  • Richard Wallace.

"Asp, a Swedish researcher who once majored in industrial design, volunteered for the fMRI probe. The scanner revealed a personality quite at odds with her own sense of self. She searched the scanner's images for the excited neurons in her prefrontal cortex that would reflect her enthusiasm for Prada and other high-fashion goods. Instead, the scanner detected the agitation in brain areas associated with anxiety and pain, suggesting she found it embarrassing to be seen in something insufficiently stylish. It was fear, not admiration, that motivated her fashion sense."

"AI is much more likely to be a boon than a threat to humans. In many ways one can best describe AI technology as the development of what my colleague Ken Ford calls 'cognitive prostheses': systems that people can use to amplify their own intellectual capacities. Such tools empower people and aid in removing social barriers. To dramatize the point: about a hundred years ago, rapid mental arithmetic was considered an impressive intellectual talent, and people who could do it received academic honors. Nowadays a high-school dropout at a supermarket checkout can tell the customer the total charge in a fraction of a second. A barcode scanner and a computer read-out act as a mental amplifier enabling someone to perform a task that, without it, would require greater mental capacity than he could deploy unaided. True, we don't usually say that the supermarket checkout clerk is using this machinery to think with; but ask yourself: who is earning the wages, the human or the computer?"

  • www.aaai.org/AITopics/html/faq2.html#hayes
  • "NaĆÆve Physics" researcher Pat Hayes replying in the AAAI FAQ Annex to a student asking about the threat posed by AI.

"A creature that was built de novo might possibly be a much more benign entity than one with a kernel based on fang and talon."

  • www.ugcs.caltech.edu/~phoenix/vinge/vinge-sing.html
  • SF writer Vernor Vinge writing about the Singularity.

"Artificial intelligence is the study of how to make real computers act like the ones in the movies."

"Yes, now there is a God."

Python for Robotics

Avoiding the Karel-the-robot paradox

In this feature, I continue last month's Python for AI by moving on to robotics and the Pyro robot-control software. Pyro's designers devised it to overcome the limitations of Lego Mindstorms for teaching. In their paper Avoiding the Karel-the-Robot paradox: A framework for making sophisticated robotics accessible, they explain who Karel the robot was and why he is to be avoided.

Karel was introduced by Richard Pattis in his book Karel the Robot - A Gentle Introduction to the Art of Programming. His book isn't on the Web, but I did find a Karel-based course for C, by Roland Untch. This introduces us to Karel, who lives in a grid of streets and walls. Scattered throughout this grid are beepers, which Karel can sense, pick up, and put down. Students learn to program by instructing Karel to perform assorted tasks, using commands such as TurnLeft() and PickBeeper(). This highly imperative style of programming is - I imagine - one that students find easy to get started with. However, the authors of Avoiding the Karel-the-Robot paradox assert that it eventually leads students to a programming dead-end. Similarly, they say, although inexpensive robots have made introductory AI accessible to a wide range of school and university students, they have led to a robotics dead-end.

One problem is portability. There are many robots on sale, but each tends to have its own programming language and development tools, often very different from those of other robots. This make it difficult for students to transfer not just code, but also programming techniques.

Also, many robot programming systems are restricted in the sensors they support. For example, many low-cost robots are often supplied with infrared range sensors only. You might be able to add something more sophisticated such as a sonar or laser range sensor; but even if your educational budget can afford this, you may not be able to access the sensor from the software.

So, widespread use of robots for teaching AI needs not just cheap hardware, but also control software that can be ported to many different robots and make them all look identical to the student. That's Pyro's goal: write-once/run-anywhere robot programs. Then students can concentrate on building robot brains. Also, as they learn, they will be able to gradually move up to more and more sophisticated robots. And such robots, if the software is capable enough - and Pyro should be - will be usable in research as well as teaching.

Pyro

Pyro is available at pyrorobotics.org/, and supports a wide range of robots: Pioneer and PeopleBot family, Khepera and Hemisson family, and Aibo and simulators RoboCup Soccer Player/Stage Gazebo and Khephera. Pyro can be used with Orocos, the Open Robot Control Software that I mentioned last month.

Pyro runs on Unix and Linux, but according to the Pyro FAQ, may also work with other operating systems. A LiveCD is available; and Zach Dodds has made a Windows implementation, PyroWin.

The Pyro library includes modules for various robot control paradigms, robot learning, robot vision, localization and mapping, and multiagent robotics. The robot control paradigms include modules for direct control, finite state machines, subsumption architecture, fuzzy logic control, and neural network control: feedforward, recurrent, self-organizing maps, other vector quantizing algorithms. There are also genetic algorithms and genetic programming. The vision modules provide a library of the most commonly used filters and vision algorithms enabling students to concentrate on the uses of vision in robot control. All this is open source: it can be modified, and students can learn by looking at the code. (The documentation is also open source, available under a Creative Commons licence.) Modules planned for the future include logic-based reasoning and acting, classical planning, and path planning and navigation.

Pyro for direct control

One Farside cartoon depicts two amoebae sitting in front of a television. The female amoeba, sporting typical Larson nagging-wife upswept glasses, is glaring at the male amoeba and shouting "Stimulus, response. Stimulus, response. Don't you ever think!". If stimulus-response control is low on the evolutionary ladder, it's also easy to teach: let's start there, with an example that's reprinted in several of the papers about Pyro, including The Pyro toolkit for AI and robotics:

from pyro.brain import Brain



class Avoid(Brain):



    def wander(self, minSide):

        robot = self.getRobot()

        # if approaching an obstacle on the left side, turn right

        if robot.get(ā€™rangeā€™,ā€™valueā€™,ā€™front-leftā€™,ā€™minvalā€™) < minSide:

            robot.move(0,-0.3)

        # if approaching an obstacle on the right side, turn left

        elif robot.get(ā€™rangeā€™,ā€™valueā€™,ā€™front-rightā€™,ā€™minvalā€™) < minSide:

            robot.move(0,0.3)

        # else go forward

        else:

            robot.move(0.5, 0)

    def step(self):

        self.wander(1)



def INIT(engine):

    return Avoid(ā€™Avoidā€™, engine)

Here, we're defining a robot "brain". These have to be subclasses of class Brain. This one is class Avoid: in Python, although it might look like some kind of procedure call, the code

class X(Y)

defines new class X to be a subclass of Y.

Every Pyro brain needs a step method, which Pyro executes on every control cycle. The one above makes the robot continually wander, turning as a direct response to its range sensor if it has got too close to an obstacle on either side.

The authors emphasise that this program does not depend on the robot or range sensor. it's also independent of the robot's length, since Pyro translates sensory and motor data to multiples of length, and will avoid obstacles when they are within one robot length of the front-left or front-right range sensors, whatever that happens to be.

Pyro for behaviour-based control

Let's move on to a robot controlled by a finite-state machine. The robot's job is a bit of simple recycling, picking up and storing cans. The authors use a simulated Pioneer robot with gripper and "blob" camera, discussed in the next section, on vision. Cans are represented as randomly positioned red pucks in a circular environment without obstacles. The robot's goal is to collect all the red cans. Once it has picked up a can, it stores it and looks for more cans.

The finite-state controller has four states: locateCan, approachCan, grabCan, and done. Each state corresponds to a particular behaviour: it is triggered by some condition in the environment, tries to handle the condition, and may then move to another state.

The controller starts in state locateCan. In this state the robot rotates, looking for a blob which would mean a red can is in sight. If it finds a can, the controller switches to state approachCan to move the robot toward the closest visible can. (If the robot loses sight of the can, the controller returns to state locateCan.) Once the robot has its gripper around a can, the controller switches to state grabCan, making the robot pick up and store the can. It then returns to state locateCan to search for another can. This state keeps track of how long it searches on each activation of the state. If the robot has done a complete rotation and not seen any cans, the controller switches to state done and stops.

Here's the locateCan state in Python. As with the direct-control brain, each state must implement the step method, called on every control cycle. States use the goto method to switch to other states:

class locateCan(State):



    def step(self):

        # get a list of all blobs:

        blobs=self.get("robot/camera/filterResults")[1]

        # checks if there are any blobs

        if len(blobs)!=0:

        # stops robot when a blob is seen

        self.robot.move(0, 0)

        print "found a can!"

        # transfers control to homing behavior:

        self.goto('approachCan')

        # checks if robot has done a complete rotation

        elif self.searches > 275:

        print "found all cans"

        # transfers control to completion behavior:

        self.goto('done')

        #otherwise keep rotating and searching

        else:

        print "searching for a can"

        # updates rotation counter:

        self.searches+=1

        # rotates robot and remains in locate behavior:

        self.robot.move(0, 0.2)

Pyro for vision

What about vision? As already mentioned, Pyro has camera-interface and image-processing modules. Students can write programs to implement vision algorithms, such as colour histograms, motion detection, object tracking, or edge detection.

For efficiency, the low-level vision library code is written in C++, but students can interactively use it to build layers of filters in Pyro, calling the computationally expensive C++ code while still having the benefits of the high-level, interactive interface of Python.

The authors illustrate with Aibo looking at a ball and applying three filters to the raw image: colour matching, supercolour, and blob segmentation. The colour matching filter marks all pixels in an image that are within a threshold of a given red/green/blue colour triplet. The supercolour filter magnifies the differences between a given colour and the others. For example, the supercolour red filter makes reddish pixels more red, and the others more black. Finally, the blob-segmentation filter connects adjacent pixels of similar colour into regions, computes a box completely surrounding the matching pixels, and returns a list of these bounding boxes. Students can use these filters without needing to worry about the low-level image-processing details - for example, detecting Aibo's ball by finding the largest region matching its colour, then drawing a bounding box around it. It's then easy to program Aibo to move towards this region.

Pyro and Aibo

If you own an Aibo - surely the most popular of Pyro's robots - why not consider Pyro as an alternative to Sony's Open-R and other development tools? As the examples from Pyro's Using the Sony AIBO Robot page, commands are not difficult to write:

robot.setPose("mouth", 1.0)

robot.setPose("tail", 0.2, 1.0)

robot.setPose("left leg front knee", 0.5)

robot.getSensor("ir near")

robot.setWalk("TIGER.PRM")

The getSensor gets data from one of Aibo's infra-red sensors, and the setWalk loads a gait.

Using the Sony AIBO Robot also mentions that two Aibo "brains" are available: one for following a blob, and one which tries to kick a ball into a goal. This indicates that, as one would expect, Aibo can be used with Pyro's software for camera control and vision.

I suspect the ball-kicking brain is that described in Ioana Butoi's dissertation Find Kick Play: An Innate Behavior for the Aibo Robot. This explains how Pyro was used to build Aibo software for recognising a ball and a goal, and kicking one towards the other. Butoi describes object-recognition algorithms developed for the RoboCup competition, and also how to stop Aibo falling over as it kicked the ball. Butoi had to devise a stance in which Aibo could balance on three legs while kicking with the fourth. A real dog might do that too (though in my experience, it's more likely either to eat the ball or bite the experimenter); but a real dog would be intelligent enough to constantly adjust its stance as its fourth leg swings and kicks. Aibo isn't that clever, so Butoi had to find a specially stable joint configuration for it to balance on.

Pyro versus Tekkotsu

Tekkotsu is an application development framework developed at CMU for Aibo and other intelligent robots. Like Pyro, it is intended for educational use: how does it compare?

Pyro developer Douglas Blank says in Using the Sony AIBO Robot and in a posting about the Pyro-Tekkotsu relationship that Aibo Pyro actually uses part of Tekkotsu, namely the Monitor - a set of servers running on Aibo via which programs can transfer sensor data, images, and motion commands. That doesn't mean students need to learn Tekkotsu, though. Blank goes on to say in his posting:

The main project of Tekkotsu offers a unique programming environment. If I were going to land an Aibo on the moon, I'd probably use Tekkotsu to control it. But for doing interactive teaching, and high-level scripting and experiments in the lab, I'd use Pyro. To give you an idea of the environments: In Tekkotsu, if you want to change a line of code, you must recompile everything that depends on the code (it is C++ code) using the provided cross-compiler. Then the code is copied to the dog over ftp, the dog shuts down, and starts back up. The whole process (compile + transfer + reboot) lasts at least a minute on our machines. In Pyro, you simply press the "reload brain" button and nearly instantly you are running the new code.

I love the idea of Aibo on the Moon.

Pyro in general
Home page for Pyro Python Robotics. Don't confuse this with Python Remote Objects at pyro.sourceforge.net/, also named Pyro.

https://www.cs.hmc.edu/~dodds/PyroWin/ - PyroWin, Pyro modified to run under Windows, by Zach Dodds. "At some point, the official version of Pyro may run under Windows out-of-the-box, and this page will disappear".

the Pyro FAQ, which answers some questions about how the software works.

emergent.brynmawr.edu/pipermail/pyro-users/2004-September/000050.html - [Pyro-users] Re: Pyro High-Level Conceptual Model.

Pyro in teaching

www.cs.hmc.edu/roboteducation/FinalPapers/Blank.pdf - Avoiding the Karel-the-Robot Paradox: A framework for making sophisticated robotics accessible, by Douglas Blank, Holly Yanco, Deepak Kumar, and Lisa Meeden. Presented at AAAI 2004 Spring Symposium.

www.mtsu.edu/~untch/karel/ - Roland Untch's C course using Karel. Not Pyro-related, but shows who the original Karel was.

dangermouse.brynmawr.edu/~dblank/papers/aimag05.pdf -* The Pyro toolkit for AI and robotics*, by Douglas Blank, Deepak Kumar, Lisa Meeden, and Holly Yanco. Submitted to AI Magazine.

The main Pyro Curriculum page. Links to course notes on Pyro for behaviour-based control, neural nets, vision, and other topics. Also links to two slide presentations: the AAAI 2005 overview (10 slides), and the AAAI 2005 tutorial (118 slides). These, particularly the tutorial, contain: examples of Python code, course topics, and student projects; defects of Lego robotics; diagrams of the Pyro architecture; pictures of the robots and simulators.

www.cs.pomona.edu/~marshall/papers/bringing_up_robot.pdf - Bringing up robot: Fundamental mechanisms for creating a self-motivated, self-organizing architecture, by Douglas Blank, Deepak Kumar, Lisa Meeden, and James Marshall. Interesting paper on self-organising maps for a hierarchical control architecture, where each level "chunks" sequences for use by the more abstract level above it.

Pyro and Aibo

Using the Sony AIBO Robot, on the Pyro site.

cs.brynmawr.edu/Theses/Butoi.pdf - Find Kick Play: An Innate Behavior for the Aibo Robot, by Ioana Butoi, Bryn Mawr, 2005.

Pyro versus Mindstorms and Tekkotsu

emergent.brynmawr.edu/pipermail/pyro-users/2005-February/000087.html - [Pyro-users] Pyro-Tekkotsu relationship ?.

Gigglebytes

Quite by chance, I found the following in a book bought second-hand from Oxfam some weeks ago:

A few years ago, Dr Graham Ritchie and Dr Kim Binsted created a computer programme that could produce jokes. We were keen to discover if computers were funnier than humans, so entered five of the computer's best jokes into LaughLab. Three of them received some of the lowest Joke Scores in the entire database. Here are those failed puns:

What kind of contest can you drive on?

A duel carriageway.


What kind of line has sixteen balls?

A pool queue.


What kind of pig can you ignore at a party?

A wild bore.

However, two examples of computer comedy were surprisingly successful and beat about 250 human jokes:

What do you call a ferocious nude?

A grizzly bare.


What kind of murderer has fibre?

A cereal killer.

So, jokes written by a computer are not particularly funny to humans, but perhaps they would be hilarious to other computers.

It's from Laughlab: The Scientific Search for the World's Funniest Joke, by the British Association for the Advancement of Science, and refers to the work linked to below.

www.laughlab.co.uk/ - LaughLab, created by Richard Wiseman, University of Hertfordshire, in collaboration with the British Association for the Advancement of Science.

A symbolic description of punning riddles and its computer implementation, by Kim Binsted and Graeme Ritchie, 1994. Early paper, explaining the theory behind such riddles as "What do you give an elephant that's exhausted? Trunkquillizers", and its embodiment in the first version of JAPE.

https://www.inf.ed.ac.uk/publications/online/0158.pdf - The JAPE riddle generator: technical specification by Graeme Ritchie, 2003. The paper contains formal definitions of JAPE-3's data structures, rules and procedures: "the aim is to set out a formally precise, implementation-independent account of how JAPE generates punning riddles. The reason for doing this is that experimental AI programs are usually under-documented, making it difficult for other researchers to replicate the work, or to know what theoretical claims are actually embodied in the implementation."

doc.utwente.nl/fid/1183 - Humour Research: State of the Art, by Matthijs Mulder and Anton Nijholt, Twente. A recent survey of humour theory and of joke generators such as JAPE, the Light Bulb Joke Generator, and Elmo, the Natural Language Robot. Includes a section on resources such as WordNet.

https://groups.inf.ed.ac.uk/standup/papers/thepsychologist_0203omara.pdf - What do you get when you cross a communication aid with a riddle?, by Dave Oā€™Mara and Annalu Waller. Also in The Psychologist, volume 16, 2003, this paper is published by the STANDUP project (System To Augment Non-speakers Dialogue Using Puns), which seeks to use humour to help language-impaired children communicate.

www.aaai.org/AITopics/html/toons.html - IAMAI's AI-toons. This AAAI cartoon page includes news on STANDUP and other humour research. It explains that "Kim Binsted had always had a love for making people laugh and was part of the improvisational comedy team at school. When her interest in physics and maths took her into artificial intelligence she fell back on her comedy background to help her work on a few problems in computers. Now, having created a programme where computers can generate their own puns, she works on a system that uses comedy to help children learn a new language, whilst still trying to fit a little improv in, in her spare time."