Week's Worth of Koby Quick-Hits May 16th to 18th 2012
Here are the quick-hit ponderings that I posted on the IBM Netezza Facebook page this past week. Clearly, I was focused on the "big" side of big data, and on the "statistics" DNA of the analytics that power big data, and on the limits of what you can in fact "optimize" with big data and analytics:
That term is starting to feel antiquated, in part because we're all focusing on using the "glut" as big data analytics fodder. And the notion of "junk" data also feels archaic, because we're starting to recognize that an individual datum, no matter how seemingly trivial in isolation, can produce rich insights when aggregated and mined with others of its kind. I'm wondering where the line between wheat and chaff data is these days.
I suppose we have to continue to make an attempt to define that line, at least for the purpose of deciding what to archive, what to purge, and what to de-duplicate.
There's this hopeful notion in the geek-o-sphere that stat, math, machine learning, and algorithms are suddenly cool and hip. Others may have different perspectives, but I'm not really seeing it. Even among the Hadoop cognoscenti, it's not as if everybody's walking around boasting of their prowess with MapReduce, modeling, multinomial logistic regression analysis, and the like. That stuff is simply taken for granted these days, if you plan to stay in the big data analytics game.
There's a big difference between what can make you rich and successful in your career vs. what makes you the life of the party.
Something inside me rebels against the notion that you can "optimize" someone's experience, as if it were a formula that you, or the person in question, could put their finger on, even if they wanted to. And I bristle at the notion that post-transaction satisfaction surveys can give you a reliable feed of intelligence on how someone truly feels. Like many people, I am very likely to ignore all requests to respond to such surveys. This is especially true when I'm happy. Having to respond is a burden that detracts from my experience. And when I'm disgruntled, I have every reason to expect that my response will ignored or "de-identified" to oblivion.
I just want some human being to connect with me ASAP and make me happy again. Is this too much to ask?
At the end of the week, I'm always focused on "gut feel" issues. Experience is that visceral gut-level context in which we, as humans, absorb this overload of "intelligence." Heart tells the head to cool it a bit.
Follow IBM Netezza On:
Follow Jim On: