Ordinary Heroes

Hancock is on the way !

Most people, even comics fans won't know about this guy. The unlikely,
anti-hero with superpowers even Superman would envy. He starts out as a
homeless bum (think it's the same in the movie. Not sure, I read the
comics) superpowers but realises his destiny and becomes a saviour.

But does it actually take superpowers to become a hero, or at least humane?

Two weeks ago, I was having lunch with Nick and Naim in Sentul and this
mid aged guy walks in the shop and sits at our table. Reeking of cheap
liquor, he proceeded with his normal sob story and asked for RM10.
Being the 'nice' guy I am, I politely refused and told to have a meal
on me. Well, he ordered a plate of fried noodles.

Unfortunately, the food was shared among his mouth, shirt, table and
floor. Obviously, his motive was to get more $ for his next round of
boozing.

How many of us have seen people begging on the streets, with their kids at
their side ? How many of us have given them the odd change, hoping that
the 20 sen that we give will allow them a better life. Only thing we never
realised is that, most of these people are part of a 'begging company'.
The beggars are dropped of by gang members at strategic points, and
picked up again at the end of the day to get the daily collections.

Hmm ... makes you think : The 20 sen you gave actually goes into the pocket of a guy sitting in an office, sucking on a kretek and living off these poor people.

The next time around : buy them food instead of giving them $.

The Killer App?

Semantic Web: What Is The Killer App

The Semantic Web has been in the making for some time and people think it is nearing maturity.
We have written about this trend extensively, with our two most notable posts being an analysis of the challenges of the classic bottom-up approach and the promise of the new top-down one. Regardless of how the Semantic Web will come about, for it to flourish it needs to hit the mainstream. There is no way that consumers will appreciate the elegance and mathematical soundness of RDF and OWL. People don't care about math, they care about utility and even more, about fun. What the Semantic Web needs, then, is a killer app.

Whatever it is, it needs to layer an understanding of semantics on top of a consumer application.

The consumer application needs to be so cool and so viral that people will be open to learning that it is powered by semantic technologies. In that case, it will be possible to further market applications as Semantic Web apps. Consumers will understand that if one Semantic Web application has potential, so might others. In math, this is called proof by induction. In marketing this is called creating a market. In
any case, it needs to be done.

In this post, we analyze several existing and potential applications of semantic technologies and look for the killer app.

Natual Language Understanding

Since the beginning, the Semantic Web has been associated with Artificial Intelligence. The idea
of representing information in structured form so that computers can "understand it" and then solve complex problems was one of the keystones of the Semantic Web vision. The problem is that
representing billions of existing web documents as RDF is a rather daunting, if not impossible task. An alternative would be to "teach" computers natural language. If an application could read the page
the way we read it and interpret what it says, the annotations would not be necessary.

Natural language processing has been the Holy Grail of AI for awhile now. However, it is a very difficult problem, because humansare born with the innate ability to understand language and we learn it not in a vacuum, but in the context of life. Certainly if we could replicate that with computers, it would be amazing and it would be the killer app. The problem is that this is not on the horizon. The Semantic Web technologies of today are not able to represent natural language in its entirety, and this is not really even their goal. Evenif we could represent each page completely, there is still the matterof interpreting structure into semantics, which is the magicthat our brain does so well and so easily.

Genie In The Bottle

Related to natural language understanding, is another idea that is not on the horizon. John Markoff called it"the perfect vacation."I call it the "Genie in the Bottle" to illustrate the impossibility of this. There is a misunderstanding about the Semantic Web which is floatingaround, which equates the Semantic Web with ability to solve really hard problems. It is simply not true.

For example, if you go to a new travel agency and ask them to bookthe perfect vacation for you, the travel agent will not be ableto do it, because she does not know you. In order to find the perfectvacation there needs to be constraints: where you've been before,who you are going with, what you like to do, what is your budget, etc.

Finding the "perfect" vacation is not a one shot deal, it is a process, which leverages iteration and memory.

True, with the Semantic Web the information is structured, but it does not mean that the computer can necessarily solve complex problems.These are two completely different things. Just because you have a map,does not mean that you know the best way to get from point A to point B. Having a map is necessary, but it is not sufficient, you need the algorithm to find the best path. There is a big difference between asking what is the capital of France and what is the cheapest airfair today to fly from New York to Paris. And the even harder question is:Where should I go on vacation next? Computers are not going to give us an instant, perfect answer to this question anytime soon, if ever. Again, this would be the killer app, it is just not likely to happen.

Semantic Knowledge Databases

So what is realistic and possible today? The first in the list of growing applications are Semantic Knowledge Databases. The two examples that we will look at here are Freebase and Twine. While Freebase is focusing on building essentially a semantic equivalent of Wikipedia, and Twine is focused on a personal semantic database,both are databases, both focus on knowledge management, and both are Wikipedia-like. The advantage of these databases over Wikipedia is thatthey represent information in a structured way and support queries. To understand the difference, take a look at the Alicia Keys page on Freebase and on Wikipedia.

At first glance they are very similar, but Freebase "knows" that Alicia Keys is a blues singer and it then knows other blues singers. For Wikipedia, blues is just another page, not a music genre. So Freebase can potentially answer a question of listing all blues singers, while Wikipedia can not.

This is certainly interesting but the question is will people care? Can the end consumer tell the difference? Unlikely. Today Wikipedia contains definitive references on a vast number of topics. Like Google, it is easy to search and find relevant information, and as a result, people are not likely to be in need of a better Wikipedia. With Twine the situation might prove to be different, because personal knowledge
management is an important problem. The first question is: Are their enough people
who want to be efficient in managing personal knowledge? I think the answer is increasingly likely to be "yes." And the second question is:Does knowing the semantics of knowledge help you build the best application? At the very least Twine has to beat del.icio.us bookmarks
and ideally needs to do for personal knowledge management what Highrise is doing for CRM.

But beyond the execution, there is still another problem. For a semantic knowledge base to be the killer app it needs to ignite imagination and capture people's hearts and minds. This is not likely to happen. We appreciate libraries, we can not live without them, but we take them for granted. Knowledge has been
commoditized thanks to Google, Wikipedia, and the blogosphere, and is perceived as abundant and unexciting. For this reason Semantic Databases are not likely to be the killer apps -- but they
might become a stepping stone towards one.

Semantic Search

An early candidate for the killer app in the semantic web category was search. First
Hakia and more recently Powerset marketed the idea that a semantic search engine, one that is based on the understanding of natural language, can beat Google. On top of having the pressure to deliver qualitatively better results, Semantic Search companies also have to, at least approximately, solve the problem of natural language understanding, which as we discussed earlier is a very difficult one.

Where things stand right now, it does not look like search is the killer app for semantics. The nderstanding of natural language does not seem to give you a noticeable edge in getting better search results. At least in the comparisons that we have performed earlier there is no major difference. The statistical algorithm deployed by Google is precise and good enough, which is why it has been the clear
leader in web search for the past 8 years. To unseat Google will require more than incremental improvement in search, it will likely take a paradigm shift and the creation of a different web experience. Below, we discuss how "discovery" could possibly take a bite out of the pie, but as of now Google's algorithm remains good and strong.

Social Graph

After Tim Bernes-Lee posted
his thoughts on the Social Graph, a discussion began on the web in
which people wondered if the Social Graph is in fact the Semantic Web.
This, however, is a gross misinterpretation of the post. The Social
Graph is not the Semantic Web, nor is it
the killer app of the Semantic Web. They are just two separate
concepts. The confusion comes from the fact that they both are Mathematical Graphs
or a Network. The underlying structure of both consists of nodes connected by links. Many things
in the nature and society are networks, so it is not surprising that meaning and people fall into this category.

If anything, it is more correct to say that the Social Graph is a
subset of the giant, all encompasing Semantic Web. Knowing how people
are connected is important in order to solve the perfect vacation
problem. After all, a perfect vacation should be taken together
with perfect friends, right? But jokes aside, the Social Graph is an
interesting and important trend for 2008, however, it is not really
related to Semantic Web.

Shortcuts

Increasingly, we are seeing a new breed of Semantic Applications, which we generalize as shortcuts.
This category includes SnapShots from Snap, BlueOrganizer and SmartLinks from AdaptiveBlue, Shortcuts from Yahoo!, and In-text search
from Lingospot.
What is common between all these technologies is that they leverage the
simple semantics of the content to deliver additional information.
In the case of Snap and AdaptiveBlue, the semantics is defined by the
URL, while Yahoo! and Lingospot perform text analysis.

Regardless of the method, all of these technologies deliver related
information via Ajax popups. That is, they leverage semantics to
pull the information from the web. This is essentially discovery or
reverse search. When the user is looking at a book there is a preview
with a brief description and the cover image, when the user encounters
a stock symbol he is presented with a stock chart, analysis and
additional links to the company, when the user is looking at a music
album there is a play button, and when the user encounters a movie
there is an ability to watch the trailer in place. The shortcuts remove
the need to search, instead, the related content from the web comes
right into the page.

Today's shortcut technologies are simple and still in their infancy,
but they are among the most successful examples of semantic
applications.
However, we can not call them the killer app for several reasons.

First, people perceive them as advertising, which is not the point.
Snap certainly made an early push into ads, but this is not a
representation of what these technologies will look like in the future.
Second, in their current implementation, all of these technologies are
utilities. For the same reason that people are not going to get
emotional about personal knowledge management, they will not be
emotional about shortcuts. Shortcuts will also be taken for granted.

Yet, shortcuts hold the most promise. With a few more iterations these technologies are going to get slicker and more precise.
They will leverage content and micro-context to reduce the amount of search. They will become more personalized based on user
behavior. And once this happens it will be a big deal.

Conclusion

We are still waiting for the killer app for Semantic Web,
something that can get viral and turn semantics into
a marketing term. Problems like natural language understanding still
remain difficult to solve, and the solutions do not appear to be on our
horizon right now. It also appears that
a semantic search engine, at least based on the ones we have seen to
date, does not have a substantial advantage over Google.
We are seeing the rise of early Semantic Knowledge Databases, but while
we expect them to get better and more interesting,
they are more likely to be the stepping stones to the killer app,
rather than the app itself.

In the mean time, we are seeing the rise of shortcut technologies,
which leverage the basic semantics of the content,
like URL and simple context analysis, to deliver relevant information,
links, and media directly into the page. While still very early,
these technologies hold the most promise because they are simple and
useful. We expect that the next generation of these technologies
in conjunction with personalization will deliver an interesting
alternative to search -- contextual discovery. We will discuss
this alternative in more detail in a future post.

Now tell us what you think the killer app for Semantic Web will be?
Which of these technologies do you think is the most promising?

The Zen Of Python

This for all the coders (actual or self proclaimed) out there :

The Zen of Python

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

** full text by Tim Peters**

Well, if you're an actual coder, you'll know what Tim is talking about. The rest of you, well .....