To Archive Or Not

To Archive Or Not

Farhad Manjoo had a good piece in the Wall Street Journal on the Forever Internet vs. the Erasable Internet.

The question he raises is whether items on the Internet should be archived indefinitely or whether we should be able to delete postings.

Manjoo uses the example of Snapshot where messages and photos disappear a few seconds after the recipient opens them–a self-destruct feature.

It reminded me of Mission Impossible, where each episode started with the tape recording of the next mission’s instructions that would then self-destruct in five seconds…whoosh, gone.

I remember seeing a demo years ago of an enterprise product that did this for email messages–where you could lock down or limit the capability to print, share, screenshot, or otherwise retain messages that you sent to others.

It seemed like a pretty cool feature in that you could communicate what you really thought about something–instead of an antiseptic version–without being in constant fear that it would be used against you by some unknown individual at some future date.

I thought, wow, if we had this in our organizations, perhaps we could get more honest ideas, discussion, vetting, and better decision making if we just let people genuinely speak their minds.

Isn’t that what the First Amendment is really all about–“speaking truth to power”(of course, with appropriate limits–you can’t just provoke violence, incite illegal actions, damage or defame others, etc.)?

Perhaps, not everything we say or do needs to be kept for eternity–even though both public and private sector organizations benefit from using these for “big data” analytics for everything from marketing to national security.

Like Manjoo points out, when we keep each and every utterance, photo, video, and audio, you create a situation where you have to “constantly police yourself, to create a single, stultifying profile that restricts spontaneous self-expression.”

While one one hand, it is good to think twice before you speak or post–so that you act with decency and civility–on the other hand, it is also good to be free to be yourself and not a virtual fake online and in the office.

Some things are worth keeping–official records of people, places, things, and events–especially those of operational, legal or historical significance and even those of sentimental value–and these should be archived and preserved in a time appropriate way so that we can reference, study, and learn from them for their useful lives.

But not everything is records-worthy, and we should be able to decide–within common sense guidelines for records management, privacy, and security–what we save and what we keep online and off.

Some people are hoarders and others are neat freaks, but the point is that we have a choice–we have freedom to decide whether to put that old pair of sneakers in a cardboard box in the garage, trash it, or donate it.

Overall, I would summarize using the photo in this post of the vault boxes, there is no need to store your umbrella there–it isn’t raining indoors. ūüėČ

(Source Photo: here with attribution to Spinster Cardigan)

Saving Iraq’s Jewish Scrolls

What a beautiful job by the National Archives and Records Administration (NARA).

In Operation Iraqi Freedom in 2003, our Special Forces looking for WMD instead discoverd thousands of ancient Jewish texts.

The texts dating from 1540 to 1970 taken from the Iraqi Jewish Community were sitting defiled in the basement of Saddam Hussein’s Intelligence HQS molding and decomposing under 4 feet of water.

The U.S. military and NARA rescued these texts and have painstakingly restored and preserved them through freezing, categorizing, condition assessment, stabilization, mold remediation, mending pages, washing, binding, and more.

Pictures of the collection of texts from Iraq before and after preservation can be found here.

The collection includes:

– A Hebrew Bible from 1568

– A Babylonian Talmud from 1793

– A Zohar/Kabbalah from 1815

– A Haggadah from 1902

– 48 Torah scroll fragments

– And much more.

On October 11, NARA will unveil an exhibit in Washington, DC featuring 24 of the recovered items and the preservation effort.

Hopefully, the collection of Jewish religious texts will ultimately be returned to the Jewish community from which it came, so that it can be held dear and sacred once again, and used properly in religious worship and never again held hostage or profaned.

Thank you so much to both the Department of Defense and to the National Archives for saving and preserving these ancient, sacred Jewish religious texts.

You did a beautiful mitzvah! ūüėČ

The Internet Lives

While the Internet, with all its information, is constantly changing with updates and new information, what is great to know is that it is being preserved and archived, so present and future generations can “travel back” and see what it looked liked at earlier points in time and have access to the wealth of information contained in it.
This is what the¬†Internet Archive¬†does–this non-profit organization functions as the Library of the Internet. It is building a “permanent access for researchers, historians, scholars, people with disabilities, and the general public to historical collections that exist in digital format.”

In the Internet Archive you will find “texts, audio, moving images, and software as well as archived web pages” going back to 1996 until today.

I tested the Archive’s Wayback Machine with my site¬†The Total CIO¬†and was able to see how it looked like back on October 24, 2010.

It is wonderful to see our digital records being preserved by the Internet Archive, just like our paper records are preserved in archives such as¬†The Library of Congress, which is considered “the world’s most comprehensive record of human creativity and knowledge”),¬†The National Archives, which preserves government and historical records, and¬†The National Security Archive, a research institute and library at The George Washington University that “collects and publishes declassified documents through the Freedom of Information Act…[on] topics pertaining to national security, foreign, intelligence, and economic policies of the United States.”

The Internet Archive is located in San Francisco (and my understanding is that there is a backup site in Egypt).

The Internet Archive is created using spider programs that crawl the publicly available pages of the Internet and then copy and store data, which is indexed 3 dimensionally to allow browsing over multiple periods of times.

The Archive now contains roughly 2 petabytes of information, and is growing by 20 terabytes per month. According to The Archive, the data is stored on hundreds (by my count it should be about 2,000) of slightly modified x86 machines running on Linux O/S with each storing approximately a terabyte of data.

According to the FAQs, it does take some time for web pages to show up–somewhere between 6 months and 2 years, because of the process to index and transfer to long-term storage, and hopefully the process will get faster, but in my opinion, having an organized collection and archiving of the Internet is well worth the wait.

Ultimately, the Internet Archive may someday be (or be part of) the Time Capsule of human knowledge and experience that helps us survive human or natural disaster by providing the means to reconstitute the human race itself.

(Source Photo: here)

Federal Register On Steroids

Now, here is a new way of looking at the information from GovPulse, a site developed to “make such documents as the Federal Register searchable, more accessible and easier to digest…to encourage every citizen to become more involved in the workings of their government and make their voice heard.” ¬†The site is built from open source.
You’ll see that there is a lot more information readily available, organized in multiple ways, and really quite user-centric; some examples:
1) Number of Entries for the Day: The number of entries for the day are listed right at the top.
2) Calendar for Selecting Day of Interest:  Next to the number of entries for the day, you can click on the calendar icon and get an instant 3 months of dates to choose from or enter another date of interest and be instantly take to there.
3) Statistics for the Day: The right sidebar displays the locations mentioned on a map and the types of entries and reporting agencies in pie charts.
4) Department Entries are Prominently Displayed: Both the number of entries for each department are identified as well as identifying their type and length along with an abstract for the entry. Each Department’s entries can easily be expanded or collapses by clicking on the arrow next to the department’s name.
5) Entries are Enabled for Action: By clicking on an entry, there are options to share it via social media to Twitter, Facebook, Digg, and Reddit to let others know about it and there is also a listing of your senators and representatives and their contact information to speak up on the issues.
Additional helpful features on the homepage–immediate access to areas that are last chance to act¬†or what’s new, such as:
1) Comments closing in the next 7 days
2) Comments opened in the last 7 days
3) Rules taking effect in the next 7 days 
4) Rules proposed in the last 7 days 
Moreover,  you have another map with bubbles showing mentioned locations or you can enter your own location and get all the entries subdivided by 10, 15, 20 miles and so on up to 50 miles away.
Another feature called Departmental Pulse, show a trend line of number of entries per department over the last year or 5 years.
At the top of the page, you can quickly navigate to entries in the Federal Register by agency, topic, location, date published, or do a general search. 
There are other cool features such as when you look at entries by department, you can see number of entries, places mentioned, and a bubble map that tells you popular topics for this department.
Overall, I think GovPulse deserves a big thumbs up in terms of functionality and usability and helping people get involved in government by being able to access information in easier and simpler ways.
The obvious question is why does it take 3 outsiders “with a passion for building web applications” to do this?
While I can’t definitively answer that, certainly there are benefits to coming in with fresh eyes, being true subject matter experts, and not bound by the “bureaucracy” that is endemic in so many large institutions.
This is not say that there are not many talented people in government–because there certainly are–but sometimes it just takes a few guys in a garage to change the world as we know it.

Federal_register Govpulse