Wednesday, November 14, 2007

Filter to put an end to Stupid Comments

I recently read an article in Fortune Magazine which claims that Gabriel Ortiz, a 27-year-old systems administrator in Albuquerque is developing a comment filter that will disallow stupid comments. Any comment would have to pass through this filter. If the filter detects that the comment does not make much sense, it would flash an alert to the poster stating that the comment is quite unintelligible and if the user would like to reword it. The user can either chose to restate it to make some sense or give up and move on to some other more lenient site. Making such a filter that disallows stupid comments is quite difficult. It is far more complex than things like spam filters which do relatively simple things like pattern matching. But this is an entirely different ball game. Simple pattern matching for abusive words would filter comments to some extent, but understanding the contextual usage of a word requires much more than simple pattern matching. Even if the algorithm can detect whether a comment makes some sense or not. It is quite impossible to detect subtleties irony and sarcasm by it. Natural Language Processing (NLP) has not yet advanced to a level where these language nuances can be detected. Ortiz claims to have solved the problem by detecting the correctness of the spellings in the comment. While intelligent people are prone to using sarcasm and irony that makes little sense to the computer, they usually spell correctly. If Ortiz's software does what it claims to do, still a certain degree of manual intervention will be required for some final decision makings. But undoubtedly the degree of intervention would decrease considerably. With the current state of detection of context by the search engines which can at best be quite bad, it remains to be seen how much Ortiz's filter will be capable of filtering out stupid comments. The project is aptly termed as Stupid Filter and you can get more information on it from its official site http://stupidfilter.org/main/

No comments:

Post a Comment

I'd love to hear from you !