Wednesday, February 15, 2012

Brain Capacity Limits Exponential Online Data Growth

In a way they are stating the obvious here. Our minds are not able to digest data past a certain intuitive level although that clearly varies.  I will take it a little farther that that.  The process of digesting date also entails properly labeling it and then hoping the brain can do the rest.

I have had great success in absorbing a mass of data over time on problems of interest that I chose to carefully define.  Observe the importance of carefully defining the problem itself.   This has led to a wide range of solutions sometimes after twenty or even fifty years.

It is thus important to inquire after the truth, but also important to set it aside for the nonce until a solution presents itself.  Discovery requires the gift of patience.

Brain capacity limits exponential online data growth

by Staff Writers

Frankfurt, Germany (SPX) Feb 10, 2012

Scientists have found that the capacity of the human brain to process and record information - and not economic constraints - may constitute the dominant limiting factor for the overall growth of globally stored information. These findings have just been published in an article in EPJ B by Claudius Gros and colleagues from the Institute for Theoretical Physics at Goethe University Frankfurt in Germany.

The authors first looked at the distribution of 633 public internet files by plotting the number of videos, audio and image files against the size of the files. They gathered files which were produced by humans or intended for human use with the spider file search engine

They chose to focus on files which are hosted on domains pointing from the online encyclopaedia Wikipedia and the open web directory dmoz. Assuming that economic costs for data production are proportional to the amount of data produced, these costs should be driving the generation of information exponentially.

However, the authors found that, in fact, economic costs were not the limiting factors for data production. The absence of exponential tails for the graph representing the number of files indicates this conclusion.

They found that underlying neurophysiological processes influence the brain's ability to handle information. For example, when a person produces an image and attributes a subjective value to it, for example, a given resolution, he or she is influenced by his or her perception of the quality of that image.

Their perception of the amount of information gained when increasing the resolution of a low-quality image is substantially higher than when increasing the resolution of a high-quality photo by the same degree. This relation is known as the Weber-Fechner law.

The authors observed that file-size distributions obey this Weber-Fechner law. This means that the total amount of information cannot grow faster than our ability to digest or handle it.

1 comment: said...

Manybe the results is not too surprising, as you state, since we are indexing only public files which are either produced by humans or intended for human use.