How do you tell the difference between art and written language?
Oh, yeah. It's math.
[Rob Lee] and colleagues Philip Jonathan and Pauline Ziman analyzed the engravings, found on the few hundred known Pictish Stones. The researchers used a mathematical process known as Shannon entropy to study the order, direction, randomness and other characteristics of each engraving.
The resulting data was compared with that for numerous written languages, such as Egyptian hieroglyphs, Chinese texts and written Latin, Anglo-Saxon, Old Norse, Ancient Irish, Old Irish and Old Welsh. While the Pictish Stone engravings did not match any of these, they displayed characteristics of writing based on a spoken language.
There is, sadly, not a lot of detail about what specific characteristics make language stand out from decoration. I'm guessing it has something to do finding patterns in the choice of symbols, or the way symbols are oriented, or how the patterns repeat. Wish there was more though. For the record, even if this is language, nobody is even close to deciphering what it means.
On a side note: Shannon entropy is a measure of the amount of information that we get from knowing one English letter. It's kind of the Entropy of Wheel of Fortune—how many guesses does it take to figure out all the letters of a sentence using only the information provided by the letters previously guessed. Besides identifying ancient scripts, it makes for a fun, time-wasting applet game.