Garbage In, Repair The World Out

Garbage Truck

I’m sure you know the saying, “Garbage In, Garbage out”–in other words what you put into something is what you get out.



In this case, I took a photo of a garbage truck–of all things–that had prominently plastered on its side, “Tikkun Olam – Repair the World.”



That is quite a positive message to put on a garbage truck!



Maybe that is our challenge in life, to make good things happen from the garbage that life often throws our way. 



Make something sweet like lemonade out of something sour like lemons.



This is not easy without some sugar, but in life, we need G-d to supply the raw ingredients and we add the elbow grease. 😉



(Source Photo: Andy Blumenthal)

>Information: Knowledge or B.S.?

>With modern technology and the Internet, there is more information out there than ever before in human history. Some argue there is too much information or that it is too disorganized and hence we have “information overload.”

The fact that information itself has become a problem is validated by the fact that Google is world’s #1 brand with a market capitalization of almost $100 billion. As we know the mission statement of Google is to “to organize the world’s information and make it universally accessible and useful.”

The key to making information useful is not just organizing it and making it accessible, but also to make sure that it is based on good data—and not the proverbial, “garbage in, garbage out” (GIGO).

There are two types of garbage information:

  1. Incorrect, incomplete, or dated
  2. Misleading /propagandistic or an outright lie

When information is not reliable, it causes confusion, rather than bringing clarity. And then, the information can actually result in worse decision making, then if you didn’t have it in the first place. This is an enterprise architecture that is not only worthless, but is harmful or poison to the enterprise.

Generally, in enterprise architecture, we are optimistic about human nature and focus on #1, i.e., we assume that people mean to provide objective and complete data and try to ensure that they can do that. But unfortunately there is a darker side to human nature that we must grapple with, and that is #2.

Misinformation by accident or by intent is used in organizations all the time to make poor investment decisions. Just think how many non-standardized, non-interoperable, costly tools your organization has bought because someone provided “information” or developed a business case, which “clearly demonstrated” that is was a great investment with a high ROI. Everyone wants their toys!

Wired Magazine, February 2009, talks about disinformation in the information age in “Manufacturing Confusion: How more information leads to less knowledge” (Clive Thompson).

Thompson writes about Robert Proctor, a historian of science from Stanford, who coined the word “Agnotology,” or “the study of culturally constructed ignorance.” Proctor theorizes that “people always assume that if someone doesn’t know something, it’s because they haven’t paid attention or haven’t yet figured it out. But ignorance also comes from people literally suppressing truth—or drowning it out—or trying to make it so confusing that people stop hearing about what’s true and what’s not.” Thompson offers as examples:

  1. “Bogus studies by cigarette companies trying to link lung cancer to baldness, viruses—anything but their product.”
  2. Financial firms creating fancy-dancy financial instruments like “credit-default swaps [which] were designed not merely to dilute risk but to dilute knowledge; after they changed hands and been serially securitized, no one knew what they were worth.”

We have all heard the saying that “numbers are fungible” and we are also all cautious about “spin doctors” who appear in the media telling their side of the story rather than the truth.

So it seems that despite the advances wrought by the information revolution, we have some new challenges on our hands: not just incorrect information but people who literally seek to promote its opposite.

So we need to get the facts straight. And that means not only capturing valuable information, but also eliminating bias so that we are not making investment decisions on the basis of B.S.

>Information Integrity and Enterprise Architecture

>

We are in an information economy and now more than ever business needs information to conduct their functions, processes, activities, and tasks.

To effectively conduct our business, the information needs to be relevant and reliable. The information should be current, accurate, complete, understandable, and available.

Information integrity is essential for enabling better decision-making, improving effectiveness, and reducing risk and uncertainty.

However, according to DMReview, 8 February 2008, “information within the [corporate] data warehouse continues to be inaccurate, incomplete, and often inconsistent with its sources. As a result, data warehouses experience low confidence and acceptance by users and consumers of downstream reports.”

“The Data Warehousing Institute estimates that companies lose more than $600 million every year due to bad information.”

What are some of the challenges to information integrity?

  1. Complex environments, [in which organizations] constantly generate, use, store, and exchange information and materials with customers, partners, and suppliers.”
  2. Accelerating change in the business environment [and] changing needs of business users”
  3. “Increasing complexity of source systems and technology
  4. Expanding array of regulations and compliance requirements

“Change and complexity introduce information integrity risk. Accelerating change accelerates information integrity risk. Compliance makes information integrity an imperative rather than an option.”

What are the particular challenges with data warehouses?

  1. Questionable input information—“Several source systems feed a data warehouse. Data may come from internal and external systems, in multiple formats, from multiple platforms.”
  2. Lack of downstream reconciliation—“As information traverses through the source systems to a data warehouse, various intermediate processes such as transformations may degrade the integrity of the data. The problem becomes more acute when the data warehouse feeds other downstream applications.”
  3. Inadequate internal controls—these include controls over data input, processing, and output, as well as policies and procedures for change management, separation of duties, security, and continuity of operations planning.

From an enterprise architecture perspective, information integrity is the linchpin between the businesses information requirements and the technology solutions that serves up the information to the business. If the information is no good, then what good are the technology solutions that provide the information to the business? In other words, garbage in, garbage out (GIGO)!

As enterprise architects, we need to work with the business and IT staffs to ensure that data captured is current, accurate, and complete, that it is entered into the system correctly, processed accurately, and that outputs are distributed on a need to know basis or as required for information sharing purposes, and is protected from unauthorized changes.

Using business, data, and systems models to decompose the processes, the information required for those, and the systems that serve them up helps to identity possible information integrity issues and aids in designing processes that enable quality information throughput.

Additionally, security needs to be architected into the systems from the beginning of their lifecycle and not as an afterthought. Information confidentiality, integrity, availability, and privacy are essential for an information secure enterprise and for information quality for mission/business performance.