Humungous Data

If you thought “Big Data” was already quite unmanageable, IEEE predicts a 1500% (x15) growth in data by 2015. That is 3 years from now.

On a similar scale, IEEE also suggests that terabit networks should be implemented soon to cater for demand in network traffic by 2015. This is up by x40-1000 times from today’s gigabit networks.

This probably also suggests that demand for data processing and delivery will need to increase by a similar scale. To some 10-40 times.

What products and skills will power the delivery of services for “Humungous Data”?

  • New Data systems – like GFS, BigTables, Hadoop, Hive, MapReduce
  • New Data patterns – No-SQL
  • Cloud computing – A must for elastic computing vs BYO data centres
  • Open data systems skills – unless you plan to pay for expensive database licenses.
  • Web Services – to tie it all together
  • Agile Architecture – often under-rated, but is increasingly important to focus corporate development.
  • Agile Security – also under-rated, but is increasingly important.

With corporations already struggling to manage data growth and demand, will this mean a growth of x15 in data staffing, or will a data specialist have to be x15 times more productive. I believe its a combination of both. New tools will make the data professional more effective. At the same time because of the lack of training and skills transfer, there will always be a need for the human bridge.

 

 

The future is indeed exciting.

About admin

Terence Chia is the administrator of this blog. He's an enterprise architect. Having been exposed to many integration and web projects, he now has exposure to a large repertoire of technology platforms. Email him for his bio.