Take the 2-minute tour ×

People search in my website and some of these searches are these ones:

qweasd qwa as
aıe qwo ıak kqw
qwe qwe qwe a

My question is there any way to detect strings that similar to ones above ?

I suppose it is impossible to detect 100% of them, but any solution will be welcomed :)

edit: I mean the "jibberish searches". For example some people search strings like "asdqweasdqw", "paykaprkg", "iwepr wepr ow" in my search engine, and I want to detect jibberish searches.

It doesn't matter if search result will be 0 or anything else. I can't use this logic.

Some new brands or products will be ignored if I will consider "regular words".

Thank you for your help

share|improve this question
I didn't get it at all.. What do you mean "like put......."? –  Kiril Kirov Jun 9 '11 at 19:14
What exactly are you trying to detect? We need more information if we're going to help. –  Dan Jun 9 '11 at 19:16
Even Google did not give any result for that put#@@ .Then What result you are giving ? :-) –  zod Jun 9 '11 at 19:16
Maybe you could put a spell checker in your search form. –  user673289 Jun 9 '11 at 19:17
There is no way to detect with a machine if a search string makes sense or not. If they enter nonsense, they will find nothing - isn't this enough? –  Tomalak Jun 9 '11 at 19:18

6 Answers 6

You could build a model of character to character transitions from a bunch of text in English. So for example, you find out how common it is for there to be a 'h' after a 't' (pretty common). In English, you expect that after a 'q', you'll get a 'u'. If you get a 'q' followed by something other than a 'u', this will happen with very low probability, and hence it should be pretty alarming. Normalize the counts in your tables so that you have a probability. Then for a query, walk through the matrix and compute the product of the transitions you take. Then normalize by the length of the query. When the number is low, you likely have a gibberish query (or something in a different language).

If you have a bunch of query logs, you might first make a model of general English text, and then heavily weight your own queries in that model training phase.

For background, read about Markov Chains.

Edit, I implemented this here in Python:


and buggedcom rewrote it in PHP:


my name is rob and i like to hack True
is this thing working? True
i hope so True
t2 chhsdfitoixcv False
ytjkacvzw False
yutthasxcvqer False
seems okay True
yay! True
share|improve this answer
+1 for a working solution. Very nice. –  John Jun 10 '11 at 16:29
+1, very impressive stuff –  user686605 Jun 11 '11 at 17:24
+1 for answering the question instead of wringing your hands and generally being a ninny like everyone else in the thread :). –  notJim Jun 16 '11 at 16:24
Implying that Markov Chains are the "background" to the technique you are using gives the impression that you are doing something much more sophisticated than you really are. The reader requires no understanding of Markov Chains to understand your solution to this. –  sanity Jul 4 '11 at 12:09
I've translated rrenaud's python script it into PHP github.com/buggedcom/Gibberish-Detector-PHP –  buggedcom Oct 25 '11 at 12:05

Assuming you mean jibberish searches... It would be more trouble than it's worth. You are providing them with a search functionality, let them use it however they please. I'm sure there are some algorithms out there that detect strange character groupings, but it would probably be more resource/labour intensive than just simply returning no results.

share|improve this answer
OK, I'll buy this. +1 –  John Jun 9 '11 at 19:33
I think I could determine if a search is jibberish pretty well by keeping a 65KB (128 * 128 array of floats) table and basically just iterating through the string. I am sure it's going to be a lot cheaper than a database query that returns no results. –  Rob Neuhaus Jun 9 '11 at 20:20
@rrenaud So you plan on doing this for every search, even the 'valid' ones? It's not worth it. –  user686605 Jun 10 '11 at 0:26
I get a 10 character input. So I add up 10 numbers, do a division, and compare to a threshold. The run time computation is super low. The biggest cost will be in collecting the data and coding it up. –  Rob Neuhaus Jun 10 '11 at 0:32
It's just a couple hours of hacking, github.com/rrenaud/Gibberish-Detector –  Rob Neuhaus Jun 10 '11 at 4:45

You could do what Stackoverflow does and calculate the entropy of the string.

Of course, this is just one of many heuristics SO uses to determine low-quality answers, and should not be relied upon as 100% accurate.

share|improve this answer
This is probably the best answer here - most people probably come to this topic in order to find a way to detect invalid nicks. –  kape123 Mar 12 '12 at 16:54

I'd think you could detect these strings the same way you could detect "regular words." It's just pattern matching, no?

As to why users are searching for these strings, that's the bigger question. You may be able to stem off the gibberish searches some other way. For example, if it's comment spam phrases that people (or a script) is looking for, then install a CAPTCHA.

Edit: Another end-run around interpreting the input is to throttle it slightly. Allow a search every 10 seconds or so. (I recall seeing this on forum software, as well as various places on SO.) This will take some of the fun out of searching for sdfpjheroptuhdfj over and over again, and at the same time won't interfere with the users who are searching for, and finding, their stuff.

share|improve this answer
Most of the visitors are kids so they just do. CAPTCHA is not a useful solution to put it before every search. Some new brands or products will be ignored if I will consider "regular words". Thank you for your help –  ahe Jun 9 '11 at 19:25
CAPTCHA is not what he needs. Why make life difficult for the users who are searching properly, seeing as jibberish searches aren't thaaat detrimental. –  user686605 Jun 9 '11 at 19:28
If that's the case, then you might do all right by throttling the searches (slightly) -- as in allow one search every 10 seconds or so. That will take some of the fun out of searching for sdfgbpoisdfbijhaoi over and over, but won't impact the people actually looking for, and finding, what they need. –  John Jun 9 '11 at 19:32
Put that in your answer... That is definitely the solution. –  user686605 Jun 9 '11 at 19:40
@chrislegend: Done. Though I do like your idea of not worrying about it maybe even a bit better. –  John Jun 9 '11 at 20:09

As some people commented, there are no hits in google for tapoktrpasawe or putjbtghguhjjjanika (Well, there are now, of course) so if you have a way to do a quick google search through an API, you could throw out any search terms that got no Google results and weren't the names of one of your products. Why you would want to do this is a whole other question - are you trying to save effort for your search library? Make your hand-review of "popular search terms" more meaningful? Or are you just frustrated at the inexplicable behaviour of some of the people out on the big wide internet? If it's the latter, my advice is just let it go, even if there is a way to prevent it. Some other weirdness will come along.

share|improve this answer
But there are hits for asdasdasdasdasdasd. In fact, there's even an asdasdasdasdasdasd.com –  BlueRaja - Danny Pflughoeft Aug 16 '11 at 22:31

If the search is performed on products, you could cache their names or codes and check them against that list before quering database. Else, if your site is for english users, you can build a dictionary of strings that aren't used in the english language, like qwkfagsd. Which, and agreeing with other answer, will be more resource intensive than if not there.

share|improve this answer
So you are suggesting he check all searches against an English dictionary? Why would anyone want to do that. –  user686605 Jun 9 '11 at 19:18
Not really, I mean small strings like "asd" or "qwe" that people often use to fill inputs. –  nosuchnick Jun 9 '11 at 19:21

Your Answer


