Introduction

Back in 1995, applets seemed so wonderful. James Gosling, one of Java's developers, recalls a demonstration he gave to a group of Internet and entertainment professionals:

As the talk began, Gosling noticed that many people were only casually paying attention. After all, what was so exciting about a new language driving a page of text and illustrations in a clone of Mosaic? Then Gosling moved the mouse over an illustration of a 3D molecule in the middle of the text. The 3D molecule rotated with the mouse movement. Back and forth, up and around. "The entire audience went 'Aaaaaaah!'" says Gosling. "Their view of reality had completely changed because it MOVED."

Ten years later, applets haven't become as popular as everyone hoped. However, there are some nice ones around, especially for teaching, and I start this month's issue with a look at what's available for demonstrating neural nets. The rest of the issue has a linguistic flavour, with a long feature on Loglan and Lojban, artificial speakable languages designed around logic; a quote from Leibniz on his dreams for such a language; and a glance at the resurrection of Terry Winograd's natural-language program Shrdlu.

Neural Nets on the Web

More than symbolic AI, neural nets seem to lend themselves to applet demonstrations - perhaps because compared with trees, graphs, and other structures, displaying the output is relatively straightforward. In this article, I want to talk about some of the applets I found on the Web.

The artificial neuron

I did a lot of searching before starting to write. The best site I found was Neural Java: Neural Networks Tutorial with Java Applets at the École Polytechnique Fédéral de Lausanne. (As usual, the list at the end of the feature shows all the URLs referred to.) This page links to a lot of demos, of which one of the best was also one of the simplest, conceptually speaking. This is the artificial neuron, which demonstrates the structure and behaviour of a single neuron. This is the biologically unrealistic kind, as used in perceptrons, back-prop, and most other neural nets.

The original applet was written by Fred Corbett - link-rot has swallowed his page - and modified by Olivier Michel and Alix Herrmann. It shows a single neuron running up the applet area, with inputs, synaptic weights, weighted sum, threshold, activation function, and output, all clearly labelled; there's a checkbox to hide the labels if you find them distracting. Text fields display the current value of inputs, weights and threshold; you can type new values into these and hit RETURN, and the applet will immediately recalculate and display new results for the following stages, without you needing to find and press the CALCULATE button.

The neuron's activation function is plotted in a neat little graph just below the output; clicking on this cycles through a range of functions - linear, Gaussian, step, sigmoid. After changing the activation function, you do need to press CALCULATE. This is, to my mind, a flaw: the controls would be more consistent if the applet recalculated immediately the graph were changed.

The colours of background, neuron, buttons, labels, and other components have been well chosen, so that they're not garish or distracting - with the possible exception of the pink arrows linking the labels to what they label.

The applet is embedded in the middle of a page which has some exercises below it, and a brief theory section above. This lists the neuron's parts - inputs, weights, and so on - and is nice because you can fit the applet and the list onto the screen at the same time. Each item in the list is linked to a glossary entry. That for "activation function" shows the graph of each function accompanied by its equation; that for "neuron output" has the formula for weighted sum. Just above the applet, there's a final link, to the instructions for use.

To bring in a bit of history, always worth doing, the Neural Java page also has a McCulloch-Pitts neuron. Simpler than the applet above, this has the nifty feature that you can toggle inputs from excitatory to inhibitory by clicking on the synapse. Unfortunately, "McCulloch" is misspelt in two places, one being the bold and large-text page heading.

Perceptrons, and biologically realistic neurons

Lausanne's perceptron builds on the artificial neuron, extending it with the ability to set the input-output pattern to train on by clicking the output column of a truth table. Training rate and number of iterations can be selected, and you can either train in one go, or single-step. As training proceeds, the applet displays the "decision hyperplane" - the line separating the two classes of inputs - evolving in the input space.

There are many other applets in this collection, including: multi-layer perceptrons; multi-layer perceptron for character recognition; a radial basis function network; and a Gaussian mixture network. Unusually - but pleasingly - amongst the sites I visited, some applets are more biologically realistic. One of these is Sebastien Baehni's retina simulator, which models the retina as a resistance-capacitance network, allowing you to set the network's parameters, draw a simple shape, and then see the result of passing that input through the retinal layers. The applet code can be downloaded, and there are links to JavaDoc documentation.

Another applet, written by Florian Seydoux, models the membrane potential of a single spiking neuron. This has an elaborate Swing-based display, which I did find slow to load. I couldn't find any explanatory text, and it wasn't immediately obvious how to start a run (an unlabelled arrow button in the top right-hand corner), but it's an interesting model.

Hopfield networks

The applets I've looked at above are one-shot networks, but Lausanne also have a Hopfield model, adapted by Olivier Michel from an applet by Matt Hill. This allows you to draw patterns on a small grid by clicking its squares and then either store them, or use them as test patterns to recall one already stored. A nice feature is that you can recall a pattern, change a few of its squares, and then use it as another test, watching the network (if your changes weren't too great) undo the damage.

This, and some of the other Hopfield applets around, demonstrate the Hopfield net acting as an autoassociator, but ignore its dynamics. This would admittedly be difficult, since the net's state has as many dimensions as the number of neurons, but it would be interesting to see someone try. The nearest I've found is an applet written by Richard DeVaul at MIT to demonstrate simulated annealing in spin glasses. This generates a set of 100-particle spin glasses, depicting each glass as a dot in two-dimensional space, with distance from the centre of a circle representing energy. As the glasses cool, one sees a group of dots moving from the energy maximum at the centre to various points on the circle's circumference.

Other sites

There are many other neural-net applets on the Web, though the Lausanne ones must be amongst the best-designed and best-explained. One of the other collections resides at Wayne State University's Computation and Neural Networks Laboratory. However, although it appears in many resource lists, the site is poorly presented, with spelling mistakes, poor text layout, dead links, and some applets whose only documentation is a research paper whose content doesn't match the applet controls. It's a shame, because some applets, such as the ball-balancer, must have needed a lot of work. Having said that, they're useful resources, although if using them in teaching, you might need to write some explanations.

Applets take a lot of time and effort to write well, so it's not surprising there aren't more around. All credit to those who have put in the hours and made their work available to the rest of us.

Links

java.sun.com/features/1998/05/birthday.html - Java Technology: the early years, by Jon Byous. "Chances are, everything you know about Java technology is only a few years old. There's a good reason for that: On May 23, 1998 the technology officially celebrated its third birthday. As part of the celebration, we interviewed several members of the Java technology team who have been around since the early days, and we put together this retrospective to share with the readers of java.sun.com. Join us on a stroll through history".

java.sun.com/products/jlf/ed2/book/ - Java Look and Feel Guidelines, 2nd edition. Web version of Sun's book on the good design of Java user interfaces, including those for applets.

diwww.epfl.ch/mantra/tutorial/english/index.html - Neural Java: Neural Networks Tutorial with Java Applets, École Polytechnique Fédéral de Lausanne. This contains the following applets mentioned above: artificial neuron; McCulloch-Pitts neuron; perceptron learning; retina simulation; membrane potential of a single spiking neuron; Hopfield model.

www.saliege.com/dynamique/projet/neurones/Hopfield.html - Gérard Chevet's Hopfield applet (in French).

web.media.mit.edu/~rich/research/mathModel/optimize/Annealing.html - The spin glass annealing applet, by Richard DeVaul, MIT.

www.itee.uq.edu.au/~cogs2010/cmc/chapters/Hopfield/ - The Hopfield Network: Descent on an Energy Surface, by Simon Dennis, University of Queensland. A nice account of Hopfield networks, their dynamics, and basins of attraction.

neuron.eng.wayne.edu/software.html - Computation and Neural Networks Laboratory, Wayne State University. This contains the ball balancing applet by Paul Watta and Mohamad Hassoun.

www.neuroinformatik.ruhr-uni-bochum.de/ini/VDM/research/gsn/DemoGNG/GNG.html - DemoGNG applet for competitive learning, by Hartmut Loos and Bernd Fritzke, Systems Biophysics Institute for Neural Computation, Ruhr-Universität Bochum.

staff.aist.go.jp/utsugi-a/Lab/Links.html - Applets for Neural Networks and Artificial Life. This site often appears in resource lists, but I haven't yet looked into it.

www.hav.com/nnhtml.htm - The HTML Neural Net Consulter from hav.Software. Two JavaScript net demonstrations.

www.statsoftinc.com/textbook/stneunet.html - StatSoft's Neural Networks page. A collection of short entries covering much that's worth knowing about problem solving with neural networks. Some examples are geared towards StatSoft's Statistica neural networks software, but the page is still a useful reference source which doesn't, as some texts do, ignore the connection with statistics. Indeed, this page is just part of StatSoft's Electronic Statistics Textbook, whose contents page at www.statsoft.com/textbook/stathome.html reveals it to be a compendium of information on everything from cluster analysis to quality control charts.

ftp://ftp.sas.com/pub/neural/FAQ.html - The comp.ai.neural-nets FAQ. Since the last modification date given is for 2002, this may be an old version (there are many versions, of varying dates, scattered around the Web), or perhaps it is no longer being maintained. Nevertheless, useful, detailed answers to questions. E.g. under "How many kinds of Kohonen networks exist?": "Teuvo Kohonen is one of the most famous and prolific researchers in neurocomputing, and he has invented a variety of networks. But many people refer to 'Kohonen networks' without specifying which kind of Kohonen network, and this lack of precision can lead to confusion. The phrase 'Kohonen network' most often refers to one of the following three types of networks: ...".

www.phys.uni.torun.pl/~duch/CI.html#Neural%20Networks - Wlodzislaw Duch's links list for Computational intelligence: numerical methods Neural networks and relevant math. An extremely thorough list: even though it has the usual proportion of dead links, there's a wealth of good material and a good starting point, covering financial applications, polyhedral computation, Reactive Tabu Search, Chernoff faces, the Euroattractor conference, and iterons, filtrons, and cellular automata. The last modification date of 18.02.2005 at the bottom of the page gives confidence it's still being cared for.

The Logical Languages Loglan and Lojban

Eskimos have 500 words for snow! In fact they don't, as I'll explain later. But the hypothesis behind this common urban myth is a serious one, and it's the same hypothesis that gave rise to Loglan and Lojban, logical languages packed with features to make you think more rationally. These, and their relevance to AI, are what I am going to talk about in this article.

The Sapir-Whorf hypothesis, or, how our language determines our thought

The 500-words myth arose from the Sapir-Whorf hypothesis, named after linguist and anthropologist Edward Sapir and his colleague and student Benjamin Whorf. A strong statement of the hypothesis says that the way we think about the world is determined by how our language describes reality. Thus, so the popular accounts go, because Russian has two words for "blue", Russians will perceive two shades of blue where we see only one; because the language of the Hopi Indians has no tenses, the Hopi anticipated Eninstein's relativistic view of the Universe as an unchanging block extending through space-time.

A different version of the hypothesis permits language to both influence and be influenced by thought - hence, the Eskimos might have all those words for snow because they live amongst, and need to work with, so many different varieties of the stuff. I'm not going to comment further - the intellectual history is complicated, and I haven't read much of what Whorf or Sapir actually wrote - but I've selected some links for the end of this article, including debunkings of the snow myth. Sapir-Whorf has generated a dreadful lot of rubbish: beware of anything that gives undue reverence to Hopi relativity, neurolinguistic programming, or the cultural relativism of scientific truth.

James Cooke Brown and Loglan

After surviving the Oxford weather for the past three weeks, I feel I have words for every kind of rain, hail, sleet and snow ever precipitated. However, rather than testing myself for the influence of weather words on thought, I've been doing what James Cooke Brown would have wanted me to do - exposing my brain to the language he invented. This is Loglan: a language intended to be as rich as any natural language, but with a unique logic-based structure.

Nowdays, Loglan interests people because of its basis in logic and its relevance to computing. That's how I shall approach it and its offshoot, Lojban. However, when James Cooke Brown began, logic was secondary. He wanted to test the Sapir-Whorf hypothesis by creating a language that would be different in some extreme way from natural languages. Making it extremely logical, rather than, say, extremely literary, just happened to be most feasible. This is how, in his book Loglan 1: A Logical Language, he explains his project:

In 1955 there was no experimental linguistics. But the then very rapid development of theoretical and mathematical linguistics seemed to be transcending the earlier descriptive stage of that science, and hence to offer clear if tentative guidelines for the construction of just such a culture-free language as my strategy required. ...

But if the constructed language were to serve as a laboratory instrument - something to "release", as it were, or at least increase the probability of observing, Whorfian effects - it would obviously not do to imitate any natural language or group of languages too closely. What was wanted, apart from smallness, was not a typical human language, but a severely atypical one. For if the Whorfian effects of second-language learning turned out to be elusive - and compared to those of primary language learning, which we had decided were impossible to isolate, we could certainly expect them to be minor - they would probably not be revealed in a single culture in a single try. In fact, we could probably expect the complete pattern of any given second language's Whorfian effects to emerge only against the background of many primary cultures, and perhaps only then after many tries. Thus a language constructed to release measurable Whorfian effects when learned as a second language should offer fairly large structural contrasts with all the first languages that might be involved. Since any natural language might eventually be involved, what was required was a diminutive but nevertheless genuine human language which was easily learned by adults and which differed from all natural languages in some scientifically interesting way. ...

The most promising way to create such a difference, it seemed to me, was to exaggerate some natural function of human language, that is, to increase the functional adequacy of some complex of linguistic structures in a way that would have a strong independent likelihood of enhancing the measurable performance of its learners on some specified set of tasks. Besides, in its original formulation the Whorf hypothesis is a negative one: language limits thought. One way of disclosing such phenomena is to take the suspected limits off, more precisely, to push them outward in some direction in which removing limits would have predictable effects. So it was settled. The diminutive language should also be a functionally extreme one in some known or presumable way: an extremely poetic one, say, or an extremely efficient one, or extremely logical.

Now there is very little scientific knowledge about the literary functions of language, and while a lot is known about efficient codes, it is hard to relate this property to Whorfian effects. Enhancing and clarifying the logical structures of the diminutive language, however, seemed to answer all the requirements of the project. There is a very considerable body of knowledge about the formal properties of logical systems; and a hyperlogical linguistic structure should have a clear and interesting Whorfian effect if it had any: namely the facilitation of certain identifiable kinds of thought. Not only that, but a language which only faintly promised such a mind-enhancing effect would almost certainly prove attractive to a large body of potential learners, namely students. Thus the idea of Loglan as a hyperlogical or thought-facilitating language had a very natural birth.

How Loglan was constructed

Brown wrote an article for the June 1960 issue of Scientific American which explains very clearly how he constructed Loglan's vocabulary and grammar. Most artificial languages intended for international communication, such as Esperanto, have vocabularies made up of natural-language roots. One of the earliest, Volapük, had a high proportion of Germanic roots: its name is from the German "Weltsprache" or "world speech", mangled to fit Volapük's sound structure. More often though, the creators of such languages have based them on Latin and Greek roots, because of their internationality.

To avoid such cultural bias, Brown started with the most frequently-spoken natural languages, which he deemed to be English, Mandarin, Hindi, Russian, Spanish, Japanese, French, and German. To make a Loglan word for some concept, he took words from each of these languages, and blended them in various ways, selecting some sounds and rejecting others. He scored each blend by adding up the proportion of sounds from each language, weighted by the language's importance: English weighted higher than Mandarin, and Mandarin higher than Hindi, because of the number of speakers. The total score for each blend measured its learnability by speakers of all eight languages, weighted by each language's importance.

As an example, Loglan for "blue" is blanu. This contains all the sounds from "blue"; all those from "lan", Mandarin for "blue"; and all those from the German "blau", if the dipthong "au" is split into its constituent vowels. But it contains only 2/3 of French "bleu", 1/2 of Hindi "nila", 1/2 of Spanish "azul", 2/7 of Russian "galuboi", and none of the Japanese "ao" or "kon". To score "blanu", each fraction was multiplied by the weighting for its language. Brown did the same for other trial words for "blue", such as blula and lablu. blanu scored highest.

Some other Loglan words which display their English ancestry are: prano ("run"); briki ("brick"); cefli ("chef"); and vegri ("green"). For what it's worth, I find these and other words pretty easy to recall.

The great Loglan-Lojban schism

Before moving on to grammar, I want to say a bit about Lojban, a younger offshoot of Loglan. Lojban users are more active on the Web and in discussion groups than Loglan users, and Lojban's Web site is bigger, with many more teaching materials. Because of this, I spent most of this article's preparation time on Lojban, not Loglan.

Lojban arose because of an intellectual property dispute. between Brown and the Lojbanists-to-be. Summarising the accounts I've read, it seems that the Lojbanists wanted to change Loglan to fix problems, and also to publish wordlists and other materials; Brown, on the other hand, regarded Loglan as an unfinished research project, and so wanted final control over the language, as well as royalties on the published materials. The dispute resulted in a court case over the name "Loglan", which Brown trademarked. Bob leChavalier, chief engineer of Lojban, founded The Logical Language Group, and Lojban went on to emerge as a complete language. In fact, it was already in use in 1987. In October 1987, leChavalier married co-Lojbanist Nora Tansky. The marriage vows were spoken in primitive Lojban.

The good news for anyone wanting to use Lojban in computing is that it has a large Web site at www.lojban.org/, with language manuals and other reference material, as well as texts written in or translated into Lojban. Be warned - Lojban uses words from its own vocabulary for grammatical terms such as "pronoun" and "sentence", even when talking about them in English. This is more precise, since its grammar is so different from that of the natural languages, but it does mean you'll need to learn the terminology before being able to make much sense of the texts.

The grammar of Loglan - names

I'll now talk about grammar. As I mentioned above, I'm going to describe Lojban, not Loglan, because more has been published about it. The grammatical structure of Lojban and Loglan is similar; they do have different vocabularies, because the Lojbanists rebuilt the words from scratch, using more accurate estimates of the numbers of speakers of world languages.

Before describing how Lojban handles predicates, I need some objects for them to relate. So like any language teacher trying to convey first impressions of a language while avoiding the complexities of noun declension, I'll start with proper names.

All parts of speech in Lojban have a specified pattern of consonants and vowels, making it possible for human and computer to parse a sentence even when they don't know what its words mean. The basic predicate words, for example, must follow the pattern CVCCV or CCVCV, as shown by dunda, meaning "give", and klama, meaning "go".

To accomodate the huge variety of names for people, cities, and everything else, Lojban restricts them less than it does other words. However, changes are often still necessary when Lojbanising a name. Moreover, Lojban doesn't allow sounds foreign to its repertoire: thus it lacks both English "th" sounds, because most people around the world can't say them. Some names, incidentally, puzzled me when I started, because they had an a where I would expect o to be the nearest Lojban vowel. I think this is because the names were translated by Americans, for whose pronounciation a was closer. This would explain why the Loglan translation of the childrens' "See Spot run!" - yes, there really is one - is Vizka La Spat.

Let's make some names then. My name, Jocelyn, becomes DJOselin. This has two unusual features: the capitals, and the full stop. The capitals indicate that the stress falls on the first syllable, contrary to Lojban's default stress rule. Lojban is designed to be audio-visually isomorphic, meaning that there's a one-to-one correspondence between the written and spoken forms of a word; thus, written Lojban always indicates stress, rather than leaving it to the imagination as in English. The full stop is not a sentence terminator: Lojban uses it to indicate a slight spoken pause, thus preventing the name running into the next word and causing ambiguity. Names which start with a vowel need a pause at the beginning as well as the end: Oxford becomes .oksfyrd.

The grammar of Loglan - predicates

Now we have some objects, let's relate them, starting with the Lojban translation of "Jocelyn goes to Oxford":

la DJOselin. klama la .oksfyrd.

The little word la looks like a French or Spanish "the", but means something more like "the object that I know by the following name". klama means "go". More precisely, klama is a predicate. Looking in the dictionary, I found it was defined as:

goes: klama: x1 comes/| to destination x2 from origin x3 via route x4 using means/vehicle x5 [also travels, journeys, moves, leaves to ... from ...; x1 is a traveller; (x4 as a set includes points at least sufficient to constrain the route relevantly)]

Like a comment in any well-documented program, the definition tells us the meanings of the arguments. The first is the actor doing the going, and the second is their destination.

Altogether, klama has five arguments, and we can fill them in as follows:

la DJOselin. klama la .oksfyrd. la kardif. la bristyl. la ferstgreituestyrn.

I've used more names here. kardif. and bristyl. are Cardiff and Bristol. ferstgreituestyrn. is my attempt at Lojbanising the train company First Great Western, which runs trains on that line. I haven't shown you how to omit arguments yet, so I needed something to fill the "via" place.

In fact, trailing arguments can be omitted just by not writing them. So

la DJOselin. klama la .oksfyrd.

means "Jocelyn goes to Oxford".
Omitting non-trailing arguments calls for more grammar. One way is to use zo'e, "something not important". So if I want not to say where I came from, I can write:

la DJOselin. klama la .oksfyrd. zo'e la bristyl. la ferstgreituestyrn.

and I can avoid naming the means of travel too, either by omitting it or by replacing it with the "something not important":

la DJOselin. klama la .oksfyrd. zo'e la bristyl.
la DJOselin. klama la .oksfyrd. zo'e la bristyl. zo'e

The apostrophe in zo'e indicates that the two vowels are pronounced separately rather than running together as a dipthong.

According to Nick Nicholas and Robin Turner in their Lojban For Beginners, most people don't write more than one zo'e. Lojban doesn't prohibit it, because such a sentence would still mean something, but it does provide other grammatical possibilities. These include the argument-position indicators fa, fe, fi, fo, and fu, which act like keyworded arguments in some programming languages. Thus,

la DJOselin. klama la .oksfyrd. fu la ferstgreituestyrn.

"Jocelyn goes to Oxford by First Great Western", where fu marks the phrase following as the fifth place.
These argument-position markers can be used in any order, so I could have written

fu la ferstgreituestyrn. klama fe la .oksfyrd. fa la DJOselin.

fa and its fellows would be very useful in Prolog when passing partially parameterised predicates to maplist and other higher-order predicates.

The grammar of Loglan - more predicates

Perhaps my subsconscious is still running on the 500 types of snow, but the first word my cursor fell on in the English-Lojban dictionary was "umbrella". This is the definition:

umbrella: santa: x1 is an |/parasol shielding x2 from x3, made of material x4, supported by x5.

So although "umbrella" is a noun, when you open it into the Lojban world, it unfolds into a predicate.

In fact, Lojban does not distinguish between nouns, adjectives, adverbs and verbs. They're all predicates. Here are some other weather-related words:

cold: lenku: x1 is |/cool by standard x2.
fall: farlu: x1 falls/drops to x2 from x3 in gravity well/frame of reference x4 [note: things can | in spin, thrust, or tide as well as gravity; (agentive "drop" = one of two lujvo: falcru and falri'a)]. hail: bratu: x1 is |/sleet/freezing rain/solid precipitation of material/composition including x2 [this is the substance, not the act or manner of its falling, which is carvi].
rain: carvi: x1 rains/showers/[precipitates] to x2 from x3; x1 is precipitation [not limited to '|'].
wet: cilmo: x1 is moist/|/damp with liquid x2.
winter: dunra: x1 is |/wintertime [cold season] of year x2 at location x3.

The entry for "hail" is so right - sleet, freezing rain, and solid precipitation is exactly what Oxford has been suffering.

The grammar of Loglan - describing objects

Loglan uses predicates to describe actions. It also uses predicates to describe objects, with the help of the word le. Here's "Jocelyn goes to Oxford by train":

la DJOselin. klama la .oksfyrd. fu le trene

where the definition of trene is

train (vehicle): trene (ren re'e): x1 is a | [vehicle] of cars/units x2 (mass) for rails/system/railroad x3, propelled by x4 [a railed vehicle or | of vehicles; also subway (= tu'unre'e), metro, trolley, tramway (= lajre'e), roller coaster; monorail (= dadre'e); cable car, sky car, ski lift (= cildadre'e)]

What's happening in this sentence is that le converts a predicate into some object I'm thinking about that the predicate describes. Imagine two robots speaking Prolog to one another. One says to the other:

?- find_referent_of_name( 'Jocelyn', A1 ), 

    find_referent_of_name( 'Oxford', A2 ), 

    find_referent_of_noun( train, A5 ),

    mark_as_unknown( A3 ),

    mark_as_unknown( A4 ),

    update_your_world_model_with_event( goes( A1, A2, A3, A4, A5 ) ).

That, roughly, is the effect the speaker wants to have on the listener. Whereas la told the listener to identify the object referred to by a proper name, le tells them to identify the object referred to by a description.

The grammar of Loglan - sets of objects and typical objects

I now want to tell some jokes:

Q: Did you know a man is knocked down on Britain's roads every eight minutes?
A: No.
Q: Yes there is, and he's getting pretty fed up about it.
Johnny's mother: Johnny told me he got 100 in his tests yesterday!
Johnny's mother: He did - 50 in spelling and 50 in arithmetic.
Teacher: Suzie, please spell "cattle".
Suzie: C-A-T-T-T-L-E.
Teacher: Leave out one of the Ts.
Suzie: Which one?
Q: Name five things that contain milk.
A: Butter, cheese, ice cream, and two cows.

All these rely on ambiguity for their effect. If any Lojbanists are reading this, I'd like to issue them as a challenge, because I suspect they'd be impossible to translate without losing the humour. Lojban is probably too precise.

The first joke, for instance, appears to be confusing two meanings of "a": "the typical", and "a particular individual". The point is that in English, "a" can indicate typicality. When I say "a Macintosh is an elegant machine", I am talking not about a specific machine - that one on my desk, or in the dealer's window - but about the typical Mac. In the same way, I might say "a typical child in England starts school at the age of five". Lojban provides a particle for such references, le'e. In general, Lojban enforces great precision in distinguishing between particular individuals, the typical individual, sets, members of sets, and masses such as sand or water.

Some general points

Rather than continuing with the grammar in detail, I'll leave the task to one of the excellent teaching texts. Instead, I want to utter a few generalities about what Lojban can do.

Lojban has equivalents for tenses and spatial adverbs and prepositions - and, being regular, treats both the same way, since location in time is the same kind of concept as location in space. It has pronouns - and distinguishes between the inclusive "we" which includes the listener, and the exclusive "we" which doesn't. And it has words for "and" and "or" - and distinguishes between "inclusive or" and "exclusive or".

Lojban also has material implication, the logical "if". But, as the following extract from What is Lojban? shows, it takes pains to distinguish this from the "if" which means causation:

Lojban connectives cannot be used to correctly translate English "If you water it, it will grow", because material implication is too weak and the special causal connectives, which connect assertions, are too strong. What can be done instead?

The English sentence "If you water it, it will grow" looks superficially like a Lojban na.a connection (material implication), but it actually has causal connotations not present in na.a. Therefore, a proper translation must involve the notion of cause. Neither the Lojban coordinating causal conjunction nor the two correlative subordinating causal conjunctions (one of which subordinates the cause and the other the effect) will serve, since these require that either the cause, or the effect, or both be asserted. Instead, the correct translation of the English involves 'cause' as a predicate, and might be paraphrased "The event of your watering it is a cause of the event of its future growing." (roda zo'u lenu do jacysabji da cu rinka lenu da ba banro)

I have known authors of expert-system knowledge bases who used logical "if" when they meant causation: that quote should be required reading for such people.

Lojban has imperatives - "come here!", "turn the light off", "please close the door". It has indicators for emotional attitude, such as surprise, interest, amusement, and love. (However, as linguist Geoffrey Sampson says in his review of John Cowan's book The Complete Lojban Language, this part of the language embodies some questionable analyses of human emotion.) And if you want to be deliberately imprecise, you can: Lojban allows you to create metaphors by combining predicates but without specifying the relationship between them. This is what I would be doing when writing "knowledge base" or "expert system" in English, were it not that we've evolved generally agreed-upon interpretations for these phrases.

You can even write poetry in Lojban, at least in the weak sense that word order is sufficiently malleable for you to rearrange words until you find an arrangement that best suits your desired rhyme, alliteration, or scansion. Ambiguities such as the line "And all the air a solemn stillness holds" from Gray's Elegy - where either the air or the stillness could be doing the holding - probably can't be translated; but then, avoiding such ambiguities is exactly why Lojban is worth study by anybody concerned with linguistic analysis.

The many implications of "no"

Before moving on to Lojban and AI, I want to look at one final topic, negation. When I started reading about Lojban, I wondered how much there could be to say. Surely building negation into Lojban was just be a matter of creating a word for "not" and adding brackets to delimit its scope? In fact, the matter is a lot richer, and should interest those working with subjects such as speech-act theory and discourse analysis.

In point of fact, Lojban does have brackets. The words ke and ke'e are used to bracket combinations of predicates, distinguishing for example "a new (freight train)" from "a (new freight) train". However, as Sampson says in his review:

In theory, standard predicate-logic notation could itself be made speakable, by assigning pronunciations to signs such as brackets and comma. But - leaving aside the fact that any standard logical system ignores many humanly-important considerations which Lojban does express, such as a speaker's emotional attitude to the propositions he states - such a language would be unusable. It would be grossly cumbersome, and would do nothing to cater to speakers' needs to foreground or suppress particular elements, or structure information into different perspectives. These things are facilitated in English by mechanisms alien to logical notation, such as the passive construction. Lojban generalizes devices such as the passive, and the contrast between forethought and afterthought sequencing ("if p then q" versus "q, if p"), to provide even more flexibility than is typical of natural languages.

Several examples are explored in Chapter 15 of the Lojban reference grammar: "'No' Problems: On Lojban Negation". The most extreme is "metalinguistic negation", where something is wrong with the statement being negated. In this case, it's meaningless to talk about its truth value; the negation can make no sense until the speaker has corrected their error. A notorious example is the sentence "I have not stopped beating my wife", when I never started in the first place. Others are "5 is not blue" - colour doesn't apply to numbers - and "the current King of France is not bald", which is meaningless because there is no current King of France.

To avoid such pitfalls, Lojban allows you, if asked "have you stopped beating your wife", to reply na'i. Your questioner will understand this not as in English - "no (and therefore, since I haven't stopped, I'm still beating)" - but as "sorry, your question was meaningless".

Another type of negation, explained earlier in the same chapter, is "scalar negation". This is exemplified by the sentence "the chair is not brown". This does negate one fact - that the chair is brown - but we also understand it as providing positive information, namely that there is still a chair, and that it does still have a colour, and that this colour is something other than brown.

Lojban has another word for "no" when you want to give this kind of answer. The problem then becomes that it could cause ambiguities. For example, if I say "I am not going to Oxford from Cardiff", does this mean I'm not going anywhere, or that I am going to Oxford but not from Cardiff, or that I am going from Cardiff but to somewhere other than Oxford? The point is that I may want to tell you that there is still some relationship between me and Oxford, or me and Cardiff, or me and movement; it's just not the one stated. Lojban provides a way to do so.

Loglan and man-machine interaction

What use could Loglan or Lojban be to computing and AI? Many science-fiction readers will have first heard of Loglan in Robert Heinlein's novel The Moon is a Harsh Mistress, in which the protagonist converses in Loglan with Mike, the intelligent computer. In fact, James Cooke Brown proposed Loglan for man-machine interaction in Loglan 1: A Logical Language. He did admit few would learn it solely for this purpose, but hoped it might become used if ever enough people had already learnt it for other reasons.

I have found one program for analysing Lojban, written, not surprisingly, in Prolog. This is Nick Nicholas's semantic analyser, written in NU-Prolog. In a paper on the program, Nicholas gives the following example of parsing mi prami lo prenu ku poi ke'a citka lo cakla: "I love the person who eats a chocolate"

q(suho(1), _FJKWJ, prenu(_FJKWJ, _FJKDO, _FJKDP, _FJKDQ, _FJKDR),

q(suho(1), _FJKGU, cakla(_FJKGU, _FJKLJ, _FJKLK, _FJKLL, _FJKLM), _FJKIQ,

citka(_FJKWJ, _FJKGU, _FJKGV, _FJKGW, _FJKGX)), prami(mi, _FJKWJ, _FJJZA,

        _FJJZB, _FJJZC))

which can be displayed more legibly as:

\exists X:

    person(X), {first restriction on X}

    (\exists Y:

    chocolate(Y); (eats(X,Y))); {second restriction on X}

    (loves(me,X)).

Loglan and machine translation

Brown proposes other applications of Loglan, including machine translation. The idea is an old and familiar one, namely that we translate one language to another by extracting the meaning of the source, representing it in a completely language-independent form, and then translating this semantic representation back into the target. The intermediate meaning representation is the interlingua or interlanguage, also known as the pivot language.

Many formalisms have been put forward for interlinguas. One of the first was Schank's Conceptual Dependency, which analyses natural language statements into primitives such as "exchange possession of", "move location of", "move bodypart", and "pass food into body through orifice". Although not invented with translation in mind, some researchers did try using it in small systems. A modern example is UNL or Universal Networking Language, developed under the auspices of the United Nations University in Tokyo. The UNL site has several presentations demonstrating small UNL translations of natural language. It also has a lot of dead links and no publications after 2002, suggesting that the project has been abandoned.

Interlinguas are not widely used. Most machine translators prefer the "transfer" method. Unlike an interlingua, this requires a separate translation program for each source-target language pair: a transfer translator for all EU languages, for example, would need an English-German translator, and a Polish-Dutch, and a Portuguese-Czech, and many others. The advantage of transfer is that it doesn't try to extract and represent the meaning of its input. Instead, it carries out a relatively superficial syntactic analysis and transformation: the kind of almost word-for-word conversion one might do in transforming the Dutch "er kwam een klop op de deur" into "there came a knock on the door". You can find out more from the introduction to Maria Flanagan's dissertation Extended Transfer Architecture for German Machine Translation.

Although transfer works moderately well for languages of similar structure, it might not for languages as different as some of those in the Pacific Rim countries. An interlingua would be more suitable. This is Nick Nicholas's view in his paper Lojban as a Machine Translation Interlanguage in the Pacific.

Loglan, RDF, and ontologies

An ontology specifies the vocabulary to be shared between programs engaged in some common task. I revealed a small, and rather military-oriented, part of OpenCyc's ontology in my AI Alphabet; there are now many others in existence. Given that Lojban has a vocabulary honed over the years for describing almost everything in everyday life in logical terms, even including Aesop's fable of The Fox and the Crow, has it been used as an ontology?

In point of fact, there seems to be very little. I did find two postings on Loglan and RDF, linked at the end. Both were by Steve Pomeroy, one proposing that to understand how to implement negation in RDF, it was worth looking at Lojban, and one giving a Lojban translation of some of the RDF used to describe the FOAF, or Friend of a Friend, project.

With what I now know of them, I think Loglan and Lojban are definitely worth considering as ready-made logical formalisms for any AI task involving linguistic analysis. As Nick Nicholas says in his Lojban page:

The attraction Lojban holds for me is that it is one formal model of language which actually puts its money where its mouth is, and tries to take on an entire linguistic system rather than toy sentences and a postage stamp's worth of vocabulary. The language's detailed and painstakingly compiled language definition materials are available, the 'write-up' has been completed, and the reference grammar has been published.

Or, to quote Sampson again:

In general, Lojban constitutes a strikingly thorough working-out of its creators' goals, and its design is responsive to a rich, subtle understanding of linguistics and philosophical logic.

I shall propose one final test: once Asterix has been translated into Lojban, we shall know it has arrived as a real language.

Links and other references

Loglan by James Cooke Brown, in Scientific American, volume 202, June 1960. A very clear explanation of why and how Loglan was constructed.

www.loglan.org/ - The Loglan Institute, with links to teaching materials and other resources. James Cooke Brown's book on Loglan, Loglan 1: A Logical Language, is at www.loglan.org/Loglan1/index.html, from where his views on Loglan for AI and machine translation are linked. The translation of "See Spot run" is at www.loglan.org/Texts/vizka-la-spat.html.

www.lojban.org/ - The Lojban site. Navigation around the site is fairly easy: good places to start are the Help page, the FAQ, and the introductory book What is Lojban? at www.lojban.org/en/publications/level0.html. This book contains the translation of Aesop's The Fox and The Crow. The chapter on negation I referred to from the reference grammar is at www.lojban.org/publications/reference_grammar/chapter15.html.

members.fortunecity.com/jeroenkuiper/Linguistics/Lojban/Dictionary/aindex.html - A draft English-Lojban dictionary. I think this is somewhat out of date, but it was OK for illustrating the principles, and faster to use than the interface to the current dictionary project at www.lojban.org/jbovlaste/.

home.bluemarble.net/~langmin/miniatures/lojban.htm - Do You Speak Logic? by William Z. Shetter. A nice little essay. His other features on linguistics and language are also fun to read.

www-106.ibm.com/developerworks/rational/library/2740.html - Using UML to understand Lojban by Jérôme Desquilbet of IBM. An unusual, but very readable, application of UML. For non-UMLers, it's also a good demonstration of UML itself.

ptolemy.tlg.uci.edu/~opoudjis/lojbanbrochure/lessons/less2changeplaces.html - Lojban for Beginners by Robin Turner and Nick Nicholas. Really nice introduction, which I used in constructing some of my examples.

www.grsampson.net/Vloj.html - The review by Geoffrey Sampson at the University of Sussex, of John Cowan's book The Complete Lojban Language.

www2.cmp.uea.ac.uk/~jrk/conlang.html - Richard Kennaway's annotated resource list for artificial languages.

www.zompist.com/kit.html - Mark Rosenfelder's Language Construction Kit pages. A fun way to learn some linguistics while reading about how to create languages with the same characteristics that natural ones have.

www.zompist.com/heinlein.html - Mark Rosenfelder's review of The Moon is a Harsh Mistress.

en.wikipedia.org/wiki/Sapir-Whorf_Hypothesis - A good Wiki entry on the Sapir-Whorf hypothesis. There are links to entries for Loglan and Lojban.

venus.va.com.au/suggestion/sapir.html and www.aber.ac.uk/media/Documents/short/whorf.html - Notes on the Sapir-Whorf Hypothesis by Lawrence Campbell and Daniel Chandler respectively.

www.sfs.nphil.uni-tuebingen.de/linguist/issues/5/5-1239.html - Postings from LINGUIST List 5.1239, 6 November 1994, on the Eskimo words for "snow". Includes a count of the snow words in the Eskimo language Yup'ik by Anthony Woodbury, University of Texas at Austin.

www.derose.net/steve/guides/snowwords/ - "Eskimo" words for snow, by Steven Derose. Reviews Geoffrey Pullum's book The Great Eskimo Vocabulary Hoax, and cites an interesting-sounding paper by Laura Martin: Eskimo Words for Snow: A case study in the genesis and decay of an anthropological example in American Anthropologist, volume 88, 1986.

www.bohemica.com/?m=catalog&s=258&a=138 - Amusing list of Czech words for which English has no equivalent, from Dominik Lukeč, Bohemica.com.

www.lojban.org/files/software/analyser - Nick Nicholas's semantic analyser for Lojban, written in NU-Prolog; there's a paper on the program at www.lojban.org/files/papers/lojban_parser_paper. Nicholas's home page is at www.tlg.uci.edu/~opoudjis/; apart from his Lojban links and translations at www.tlg.uci.edu/~opoudjis/Play/lojban.html, there's some useful material on rendering Greek in Unicode, and - oh gosh - Hamlet translated into Klingon.

www.cs.mu.oz.au/~lee/src/nuprolog/ - The NU-Prolog home page.

gollem.science.uva.nl/SWI-Prolog/mailinglist/archive/old/1298.html - A posting by Richard O'Keefe explaining how when annotations, used in Nicholas's analyser, affect the order in which goals are called.

sow.lcs.mit.edu/2004/proceedings/Speer.pdf - Meeting the Computer Halfway: Language Processing in the Artificial Language Lojban by Rob Speer and Catherine Havasi, MIT. Short paper describing a Lojban question-answering program written in Python, probably at a very early stage in the project.

swpat.ffii.org/xatra/lojban/index.en.html - EU Patents in Logical Language!, on the Foundation for a Free Information Infrastructure site. The FFII lobbies against European software patents - see, for example, their webshop.ffii.org/ page, which demonstrates that many typical components of a typical Web shopping site are patented. They propose patents be published in a language such as Lojban, as a starting point for unambiguous translation into the EU languages. (They also say that development of software tools for logical languages would be harmed if that software could be patented.)

www.cs.cf.ac.uk/Dave/AI2/node69.html - Notes on Conceptual Dependency, from Cardiff School of Computer Science.

www.fitug.de/debate/9811/msg00297.html - Posting to the UNL list briefly comparing UNL and Lojban.

www-clips.imag.fr/geta/User/wang-ju.tsai/unllinks.html - The UNL Center page. Most of the presentations and PDFs contain samples of UNL.

www.cs.tcd.ie/courses/csll/flanagmr0001.ps - Extended Transfer Architecture for German Machine Translation by Marian Flanagan, Trinity College Dublin. This describes a Prolog implementation of the so-called "shake-and-bake" transfer method. It also explains the different translation methods, and looks at how some of today's systems work.

citeseer.ist.psu.edu/27803.html - Lojban as a Machine Translation Interlanguage in the Pacific by Nick Nicholas.

www.rickharrison.com/language/mtil.html - On the unsuitability of "logical languages" for use as interlinguas in machine translation, by Rick Morneau.

norman.walsh.name/2004/04/02/notinrdf - Not in RDF: "There isn’t really a simple "not" operator in RDF. Nevertheless, it’s useful, particularly for establishing default values. So what can we do?". An essay by Norman Walsh on implementing "not" in RDF, taking into account the difference between open-world and closed-world views of negation. The replies on this page include Steve Pomeroy's suggestion to look at how Lojban describes negation.

www.w3.org/2000/10/swap/doc/cwm - Cwm main page. Cwm is a forward-chaining reasoner for processing RDF, used in Walsh's essay.

www.w3.org/DesignIssues/Notation3.html - W3C's Notation 3 page. Notation 3, or N3, is a superset of RDF, and easier for humans to read and write. Walsh uses it with cvm in his essay.

www.w3.org/TR/rdf-primer/ - W3C's RDF primer. A good introduction.

staticfree.info/blog/lang/lojban2rdf.comments - The other posting by Pomeroy, a Lojban translation of some RDF used to describe the FOAF - Friend of a Friend - project.

freshmeat.net/projects/jbofihe/ - Main page for jbofihe, a parser for checking the syntax of Lojban text. It also translates Lojban into pseudo-English - Pomeroy uses it to translate his Lojban translation of RDF.

www.ainewsletter.com/newsletters/aix_0501.htm#o - My AI Alphabet entry for OpenCyc, with a brief excerpt from the OpenCyc ontology.

www.aaai.org/AITopics/html/ontol.html - AAAI Ontologies page.

www-ksl.stanford.edu/kst/what-is-an-ontology.html - What is an Ontology? by Tom Gruber, Stanford.

www.asterix-international.de/asterix/collection2.shtml - The languages of Asterix.

Leibniz's Calculus of Reason

... Whence it is manifest that if we could find characters or signs appropriate for expressing all our thoughts as definitely and as exactly as arithmetic expresses numbers or geometric analysis expresses lines, we could in all subjects in so far as they are amenable to reasoning accomplish what is done in Arithmetic and Geometry.

For all inquiries which depend on reasoning would be performed by the transposition of characters and by a kind of calculus, which would immediately facilitate the discovery of beautiful results. For we should not have to break our heads as much as is necessary today, and yet we should be sure of accomplishing everything the given facts allow.

Moreover, we should be able to convince the world what we should have found or concluded, since it would be easy to verify the calculation either by doing it over or by trying tests similar to that of casting out nines in arithmetic. And if someone would doubt my results, I should say to him: "Let us calculate, Sir," and thus by taking to pen and ink, we should soon settle the question.

I still add: in so far as the reasoning allows on the given facts. For although certain experiments are always necessary to serve as a basis for reasoning, nevertheless, once these experiments are given, we should derive from them everything which anyone at all could possibly derive; and we should even discover what experiments remain to be done for the clarification of all further doubts. That would be an admirable help, even in political science and medicine, to steady and perfect reasoning concerning given symptoms and circumstances. For even while there will not be enough given circumstances to form an infallible judgment, we shall always be able to determine what is most probable on the data given. And that is all that reason can do.

Now the characters which express all our thoughts will constitute a new language which can be written and spoken; this language will be very difficult to construct, but very easy to learn. It will be quickly accepted by everybody on account of its great utility and its surprising facility, and it will serve wonderfully in communication among various peoples, which will help get it accepted. Those who will write in this language will not make mistakes provided they avoid the errors of calculation, barbarisms, solecisms, and other errors of grammar and construction. In addition, this language will possess the wonderful property of silencing ignorant people. For people will be unable to speak or write about anything except what they understand, or if they try to do so, one of two things will happen: either the vanity of what they advance will be apparent to everybody, or they will learn by writing or speaking. As indeed those who calculate learn by writing and those who speak sometimes meet with a success they did not imagine, the tongue running ahead of the mind. This will happen especially with our language on account of its exactness. So much so, that there will be no equivocations or amphibolies, and everything which will be said intelligibly in that language will be said with propriety. This language will be the greatest instrument of reason.

I dare say that this is the highest effort of the human mind, and when the project will be accomplished it will simply be up to men to be happy since they will have an instrument which will exalt reason no less than the Telescope perfects our vision. It is one of my ambitions to accomplish this project if God gives me enough time. I owe it to nobody but myself, and I had the first thought about it when I was 18 years old, as I have a little later evidenced in a published treatise (De Arte Combinatoria, 1666). And as I am confident that there is no discovery which approaches this one, I believe there is nothing so capable of immortalizing the name of the inventor. But I have much stronger reasons for thinking so, since the religion I follow closely assures me that the love of God consists in an ardent desire to procure the general welfare, and reason teaches me that there is nothing which contributes more to the general welfare of mankind than the perfection of reason.

Links

www.rbjones.com/rbjpub/philos/classics/leibniz/meth_math.htm - The Method of Mathematics by Gottfried Wilhelm Leibniz, Preface to the General Science. From RBJones.com, "The home of metaphysical positivism".

The Resurrection of Etaoin Shrdlu

One program that would have been easier to write if we typed at computers in Lojban is Shrdlu, Terry Winograd's natural-language understander. Every AI textbook carries a reproduction of Shrdlu's sparse virtual world, a schematic robot gripper poised above a collection of wire-frame pyramids and cubes standing on a tabletop. Under this will invariably be found a sample of Shrdlu dialogue -

Person: PICK UP A BIG RED BLOCK. 
Computer: OK. (does it) 
Person: GRASP THE PYRAMID. 
Computer: I DON'T UNDERSTAND WHICH PYRAMID YOU MEAN. 
Person: FIND A BLOCK WHICH IS TALLER THAN THE ONE YOU ARE HOLDING AND PUT IT INTO THE BOX. 
Computer: BY "IT", I ASSUME YOU MEAN THE BLOCK WHICH IS TALLER THAN THE ONE I AM HOLDING. 
Computer: OK. (does it)

Shrdlu became famous as the example of pronoun disambiguation and procedural semantics. To quote Winograd:

The system answers questions, executes commands, and accepts information in an interactive English dialog... The system contains a parser, a recognition grammar of English, programs for semantic analysis, and a general problem solving system... It can remember and discuss its plans and actions as well as carrying them out... Knowledge in the system is represented in the form of procedures, rather than tables of rules or lists of patterns.

Now the Semaphore Corporation have put together a collection of links to Shrdlu's creators and their reminiscences and source code. For this and other information, including Winograd's account of how Shrdlu gained its name from Linotype by way of Mad magazine, follow my links.

Logic, Rules and Spreadsheets

An early beta version of an Excel add-in that supports rules and logic is available for those interested in exploring the integration of inferencing and spreadsheet calculation and providing feedback on initial directions of the product. If you are interested, send an e-mail to Dennis.

Links

www.semaphorecorp.com/misc/shrdlu.html - SHRDLU resurrection, from Semaphore Corporation.

hci.stanford.edu/~winograd/shrdlu/ - Terry Winograd's Shrdlu page, with dialogue and an original screen display showing Shrdlu at home in its blocks world.

hci.stanford.edu/~winograd/shrdlu/name.html - Winograd on How SHRDLU got its name.

www.poplog.org/docs/popdocs/pop11/teach/msdemo - TEACH MSDEMO, by Richard Bignell and Aaron Sloman. The teaching notes for MSBlocks, a mini-Shrdlu written in the Poplog programming language. One of many AI tutorials distributed with Poplog.

www.j-paine.org/students/practicals/popbeast/popbeast.html - AI and PopBeast. Students' practical notes I once wrote describing my Prolog-based AI which lived in a virtual world and interfaced simple language-understanding to a planner. Another mini-Shrdlu, demonstrating planning, referent disambiguation, and propositional representations.

Until next month.