Thinking about what they're calling the "Insight Economy"

Here's one definition of what technologists are calling Insight, which is a kind of ill-defined computer-assisted knowledge.

Insight may be reducible to knowledge as such. If it can be held in your head, it's knowledge, certainly. It may be actionable, too, and that's always nice.

But our fumblings around the word insight might also be our unconscious attempt to name a new and unusual _property_ of some knowledge. It has to do with the path that knowledge took from measurement to data to information to knowledge.

The information systems we've worked up over the last 150 years help us to gather information about events and states we don't, and can't, directly witness, because of things like
(a) distance -- I'm not near my shipping containers, and I can only be in one place
(b) time -- there are so many shipping containers, and I have finite time
(c) scale -- many events that are important are too small or large, or too slow or too fast, for me to observe
(d) political and other barriers

Our knowledge paths overcome these and other barriers, to reach out into reality and gather data points that serve us.

But much of the knowledge we've gathered has been of the _same_kind_ or of the _same_order_ as we would gather if we were there ourselves, doing it ourselves, climbing the knowledge-making ladder alone. The questions we ask of our systems are bite-size -- what is the stock level? What is the temperature? What is the flow? Where are the people? What is the record of events? What is the news? What is new? Sometimes we get a large answer; then we set about breaking it down into consumable pieces.

As we ask those bite-size questions of our systems and our world, we intuitively shape them to our own ability to process the answers we expect to get back. This, I think, crucially limits our information gathering -- and that limits our knowledge.

I might ask three people, "What happened at the fight you saw?" -- or look at three different videocamera feeds of the fight. But if I were at Gettysburg just after it happened, would I ask 250,000 soldiers, "What happened in the battle?" The question would be almost meaningless, because just listening to the answers would take half of my lifetime. (Similarly, it's hard to prosecute someone like Saddam Hussein because it's very hard to ask and answer the question "What happened in Iraq over the last 15 years?" -- even though almost everyone, by definition, was there!) We don't normally relate to knowledge in that godlike way, because we have good, non-godlike intuitions about what kind of knowledge-gathering is reasonable and necessary for our primate lives.

But now, with computing tools, some of those intuitions will need to be unlearned. I think there are probably a lot of What-happened-at-Gettysburg kind of questions that we are starting to be ready to ask.
We might define insight -- or some incipient knowledge-form, call it George -- as a 3-step process:
1. gathering knowledge far beyond human scale, without explicit human direction of what we are looking for (sensors at the battle, at all scales)
2. processing it without human prompting, but according to human priorities (working on the question "What happened at the battle?", at all scales)
3. distilling it to human scale, for human use (presenting various, consumable, versions of what happened. Humans sometimes are interested in: patterns, anomalies, correspondences, contradictions, singularties, lulls, etc. )

----------------------

One quick example of a system where new information capabilities are put to awkward use because of the limitation of human knowledge-making: security guards watching a bank of camera feeds.

10 years from now, say we connect those camera feeds to a "smart" monitoring system instead. The useful knowledge output of the system might be the same as that of the security-guard room: some superior authority is advised to come have a look. But the knowledge that led to that alert was never held in a human mind, and in fact, no single mind might be able to hold all the variables that our imaginary "smart" algorithm considered as it worked. No human might know what the algorithm "watched for". No human might understand what, exactly, the learning algorithm actually learned when it trained. (A real human mind might hold a completely different set of variables, have different training, and work along a different path, even as it came to similar conclusions.)

Even if you don't like my examples, perhaps you can see what I'm trying to describe anyway. Just as we send robots into terrains we can't physically withstand, we send knowledge-creation algorithms into territories we can't cogitate -- at least, not at scale -- and maybe we can't even tell them in advance what to look for.

I'm _not_ talking about spreadsheets, or the fact that computers can hold more bits than I can, or add numbers faster; I'm talking about the fact that today's machines enable a new class of knowledge creation processes which aren't intuitive because they violate our intuitions about knowledge-gathering. We need tools that will help people harness (if not understand) this new sort of knowledge-creation.

(I think simulation is another good example of a system that works to produce this new, hard-to-think-about kind of knowledge. We have all considered the question of whether, when you have simulated a real system, you have some understanding of that real system. Well, under the schema I'm setting out, we would happily say, "No, not at all, but that doesn't stop you from doing the things you want to do -- in fact, your limited understanding defines the tool's purpose.")