Grammar and where AI went wrong — an interesting discussion on


They are discussing this famous interview: Noam Chomsky on Where Artificial Intelligence Went Wrong

A few extracts from the thread:

Chomsky critiqued the field of AI for adopting an approach reminiscent of behaviorism, except in more modern, computationally sophisticated form.

Chomsky argued that the field’s heavy use of statistical techniques to pick regularities in masses of data is unlikely to yield the explanatory insight that science ought to offer.

For Chomsky, the “new AI”—focused on using statistical learning techniques to better mine and predict data—is unlikely to yield general principles about the nature of intelligent beings or about cognition.


Mark Atkins —

I agree wholeheartedly with Chomsky! In my experience, the moment you start relying on statistics in any part of a designed AI system, you’ve already gone astray.

The only exception of which I can think is when preprocessing original raw data like for smoothing purposes, before the data actually hits the analyzers / feature detections, in which case the raw data isn’t something of which the intelligent part of the system is yet aware.

Therefore I have a low opinion of Bayesian nets and data mining (the trend which Chomsky called “new AI”), both of which are among the top five trendy directions I see in AI.

My reasoning is that data do not tell cause-and-effect. This is an extremely important thing for *everyone* to realize.

I detest seeing the general public constantly misled by politicians, for example, by statistics, which happens all the time to promote some group’s specific political agenda.

That’s why it’s so easy to “lie” using statistics (“lies, damned lies, and statistics”).

I’ve also seen too many awful injustices enacted by foolish people who lacked a critical reasoning skill due to lack of awareness of this piece of knowledge/wisdom, and I’ve also seen some very intelligent people make fools of themselves for the same reason.

Intelligence is all about *understanding* and prediction, and statistics does neither.

To me, statistics is a last resort for fields like psychology where the first step is merely to notice if two things are correlated, where the phenomena are so complex that cause-and-effect is too difficult to determine right away.

But it should only be a first step for detecting a pattern and a beginning to think about possible hypotheses, at best *maybe* eventually a heuristic, but definitely not a dependable rule or physical law.


C R Hunt —

I read an interesting paper a while ago that discussed how babies learn language.

They made the point that young children can pick up new grammar from just a single example. (They cited experiments to test this.)

And that a single counterexample can suppress the over-generalization of that new grammar.

For example, linking an object and verb with a certain preposition. And then learning that this same preposition isn’t used with another similar verb.

Obviously, humans aren’t using large data sets to generate rules, and yet the rules we generate can be quite accurate.

On the other hand, I wonder if we don’t discount the shear amount of raw data we are exposed to.

For example, after seeing a cat once, a human will not confuse it with a dog. However, we aren’t just seeing one instant of a cat, are we?

We see it in motion—from many angles, and how it moves from one position to another.

How do we quantify the amount of data that we are exposed to? And how much is required to uniquely identify a cat in the future?


Merlin —

Google would disagree.

In the “Introduction to Artificial Intelligence” course (now offered by Udacity), given by Peter Norvig and Sebastian Thrun, most of the problems were translated into a “Bayesian” or search problem.

Peter Norvig is Director of Research at Google Inc. He is also a Fellow of the American Association for Artificial Intelligence and the Association for Computing Machinery.

Norvig is co-author of the popular textbook Artificial Intelligence: A Modern Approach.

Sebastian is best known for his research in robotics and machine learning, specifically his work with self-driving cars.


Mark Atkins —

>> Google would disagree.

So would eBay. So let’s talk about an experience I had with eBay’s automatic recommendation system…

One year I was buying a lot of memorabilia from Disneyland on eBay.

Of course eBay’s statistical software had no trouble recognizing the theme of Disneyland in my purchases and recommending other items I might like, and it did a very good job of that.

But I also bought a number of items that seemed unrelated: a postcard of a restaurant in San Juan Capistrano, a postcard of orange groves in Irvine, and so on.

So eBay began recommending postcards of orange groves in Florida to me.

Its “logic” (which was just statistical clustering) was apparently that I was interested in three different topics: Disneyland, a restaurant, and orange groves.

Of course another pattern is that all three of the locales were in Southern California, and eBay’s software eventually must have picked up on that statistical pattern, too, since it began recommending items from Knott’s Berry Farm, among other things.

However, in general I had no real interest in orange groves in Florida or Knott’s Berry Farm. There was in fact a very clear-cut pattern in my purchases but eBay never caught it: in general I was buying memorabilia only from the route along the I-5 freeway between San Diego and Anaheim, which was the standard route my family and I used to take to Disneyland.

A really intelligent piece of software would have had more knowledge of the world in the form of maps, and would have noticed that my purchases were consistently of places that could be plotted along a clear-cut segment of a very prominent line on a map.

That would have been truly intelligent, and it would have immediately hit 100% prediction accuracy in suggestions of items that might interest me.

Instead, all it had were three clusters that were statistically unrelated in either topic or locale, based on my rejection of its other suggestions based on locale, so overall its prediction (based on its lack of understanding, based on its limitation to statistics) was weak.

I can probably give numerous other examples of how statistics fails miserably to produce intelligent reasoning, but I thought the above example was particularly clear-cut.

Therefore I’ll keep my bias against statistics-based “intelligence”, regardless of the stature or pay of Google’s employees, thank you.

Published in: on 24/12/2012 at 11:55  Comments (2)  

The URI to TrackBack this entry is:

RSS feed for comments on this post.

2 CommentsLeave a comment

  1. Hmmm, I have lectured on Chomsky (whom I have never liked) for many years. He is quite a bit of a behaviorist himself – notice that he has never either understood or paid attention to the importance of semantics, the CONTENT of utterances. Everything is syntax to him – i.e he translates everything into “expression”.

    So when Chomsky criticises people for being behaviorist, it smacks of a petulant child, complaining about his parents but carrying on the worst of their mistakes – the worst, the most deep-rooted, because they are the most subtle, most difficult to recognise. I don’t for a moment thing Chomsky has been anything but a misfortune for linguistics. Now, thank heaven, he is out of it (more or less, none of us can live forever), and more fertile ways of thinking and discussing have gained ground against the frightened, linguistics-politically correct gobbledy-gook echoing Chomsky whatever he “suggested”. Even in the US there are far more fertile milieus.

    • I don’t know much about Chomsky (actually I haven’t read anything related to him besides this interview) but what he says here about artificial intelligence is true — common misuse of statistics is killing science nowadays because statistics just gives numbers but doesn’t go into deep understanding of the underlying processes. The last comment that I cited (by Mark Atkins about eBay) illustrates this very well.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: