Clifford Thomson sent me a link to a talk Dan Solove gave at Google on his new book The Future of Reputation. I interviewed Dan on Technometria a while back about his earlier book The Digital Person.

Dan's a very interesting speaker and raises important issues in his books and in this video. This is well worth watching if you're interested in the intersection of privacy and reputation in the Internet age.

Tags: itconversations reputation identity video privacy

One of the consequences of IT Conversations leaving Gigavox Media and returning to the Conversations Network fold is that eventually we needed to remove 'gigavox' from the feed name. Unfortunately, Feedburner will only forward a feed for 30 days and many RSS readers don't seem process permanent redirects well (change the URL permanently, not just follow it).

As a result, you might not be seeing updated IT Conversations shows in your favorite podcatcher. So, take a few minutes and make sure you're using this URL in your feeds:

http://feeds.conversationsnetwork.org/channel/itc

I checked the feed URL on my copy of iTunes and it seems to have dealt with the permanent redirect just fine (click on the "i" icon at the far right of the podcast name in iTunes to see info related to that podcast).

Tags: itconversations

This week’s Perspectives is a two-parter: an interview and companion screencast on the topic of cluster computing in the classroom. The interview is with Kyril Faenov, the General Manager of the Windows HPC (high performance computing) unit, and the screencast is with Rich Ciapala, a program manager for Microsoft HPC++ Labs.

The project demonstrated in the screencast, and discussed in the interview, is called CompFin Lab. It’s a system that enables professors to in turn enable their students to run computationally expensive financial models on large quantities of data. From the student’s perspective, you go to a SharePoint server, select a computational model, pick a basket of stocks, and run the model. Behind the scenes the task is partitioned and sprayed across a cluster of computers, then the results are gathered and presented in an Excel spreadsheet.

From the professor’s point of view, some .NET programming is required. But a framework abstracts the mechanics of dealing with the cluster, so the professor can focus on the logic of the model itself.

There are couple of key points about the evolution of high-performance computing that I want to highlight here. First, there’s what Kyril calls “the gravitational pull of data.” Increasingly, people and organizations are building vast repositories of data that other people and organizations will want to analyze in computationally expensive ways. It’s great to have access to a compute cluster in the cloud that can do the heavy lifting, but when datasets get really big you get bottlenecked trying to send the data to where the code runs. At a certain point you’d rather send the code to where the data lives.

A second and related point is that in our current model for large-scale cloud-based computing, there are only a handful of what I call intergalactic clusters — namely, those operated by Google, Yahoo, Amazon, and Microsoft. These are one-of-a-kind behemoths. You can’t replicate one of them locally and apply it to your terabytes of data. So as Kyril and his team build out their cloud-based HPC services, they’re working to ensure the services can be replicated locally.

Maybe the most optimal thing is for you to stand up a 1000-node cluster with each node having a terabyte of disk. We want to enable that. We want to be able to tell our customers: Here’s how we run this large-scale data-driven HPC applications, and here’s how, within a day or two, you can stand up one of these yourself.

The idea is that if you build one of those for your own terabyte trove of astronomical or climatalogical data, you can run your own computations against that data, and you can also share that capability with other people and organizations who want to run their code against your data.

Yesterday I posted Jane McGonigal's talk from ETech 2007 on creating alternate realities. This is the first show on IT Conversations that features our new slideshow tool for playing audio sync'd with the slides. For some talks this can make a real difference in the quality since they rely on the visuals so much. In the past we've sometimes not published good content because it relied too much on the slides. No more! Check it out and let us know what you think.

Tags: itconversations etech

A while ago I wrote an alternative search and navigation interface to InfoWorld.com. The search is broken now because the underlying engine switched from Ultraseek to Google, and nobody has updated the search wrapper. But the navigation piece still works, and while it does, I want to invite some commentary because I’m thinking of doing something similar for another project.

In this model the navigation is metadata-driven, and supports views like:

InfoWorld stories tagged ‘Silverlight’

InfoWorld news stories tagged ‘Silverlight’

InfoWorld news stories by Elizabeth Montalbano tagged ‘Silverlight’

Every piece of metadata in the tabular display is active, and toggles a filter for that item. This works especially well for the tags, and enables you to cruise through the tagspace in a fluid way. For example, try this progression:

1. InfoWorld news stories tagged ‘Silverlight’

2. Click ‘flash’ to toggle it on

3. InfoWorld news stories tagged ‘Silverlight’ and ‘Flash’

4. Click ’silverlight’ to toggle it off

InfoWorld news stories tagged ‘Flash’

The same principle holds for other bits of metadata, like storytype. So for example:

1. InfoWorld news stories tagged ‘Silverlight’

2. Click ‘News’ to toggle it off

3. InfoWorld stories tagged ‘Silverlight’

4. Click ‘Review’ to toggle it on

5. InfoWorld Reviews tagged ‘Silverlight’

6. Click ‘Martin Heller’ to toggle it on

7. InfoWorld Reviews by Martin Heller tagged ‘Silverlight’

8. Click ’silverlight’ to toggle it off

9. InfoWorld Reviews by Martin Heller

It’s powerful to explore things this way, but if I did something like this again, I’d look for ways to make these filter progressions more intuitive and discoverable.

I just don’t think people expect every item to work as a control as well as an information display. And because they don’t, it may be a bad idea to do things that way. Or maybe it’s a good idea that’s still in search of its perfect expression. I’d be curious to know what you think.

I just got back from my precinct caucus meeting (Republican) where I was elected by my friends and neighbors as precinct Vice Chair for the coming two years. I was conducting the meeting as Chair.

I love caucus meetings. This is true grassroots politics. Everyone asking question, voting for delegates to the state and county conventions, paper ballots. There's some good discussion and great participation.

I'm always shocked at how few people, relatively speaking, attend. We had a better turn out than I ever remember, but there were still only around 50 people there out of around 2000 in our precinct. Just showing up gives you a huge voice.

As for me, I ran on a platform of making my vote for congressional representative in Utah's Third district contingent on the degree to which the candidate was willing to take the Change Congress pledge. I explained the four points of the pledge, why I thought they were important, and how while I don't believe that there's general corruption problem in Congress, that I do believe money changes things and we need to find ways to reduce its influence. I felt like there was overwhelming support for that stance.

Tags: politics utah

To prepare for an interview with Tim Spalding, the founder and lead developer of LibraryThing, I re-registered with LibraryThing, spent some quality time with the service, and was wildly impressed.

At one point in the interview, Tim asked me how I, Mr. LibraryLookup, as likely a person as there is to use and appreciate LibraryThing, could have gone so long without hooking up with it.

I think part of the answer is hidden in the first paragraph: I had to re-register for the service, which I had tirekicked a year or two ago. The friction of joining and re-joining online services has become a major barrier.

There’s also conceptual friction. LibraryThing is a deep application that does lots of things, but on the surface, it appears to be a mechanism for cataloging books that you own. In fact it isn’t only that, you can just load it with books that you’ve read, or might read, as a way to seed discovery and recommendation.

Finally, there’s data friction. There are bibliophiles who will obsessively catalog their own collections, but I’m not one of them. I do, however, maintain a list of books on my Amazon wishlist. I syndicate that list to the version of LibraryLookup that alerts me when books on the wishlist become available in my local library.

What I needed was a frictionless way to reuse that list. And on this go-round with LibraryThing I found it. Sort of. You can import your Amazon wishlist into LibraryThing, which is a great way to jumpstart the discovery and recommendation process. It doesn’t yet syndicate from Amazon, so the initial import won’t be refreshed, but Tim says that’s coming.

It turns out not to matter at all that list of books I’m interested in happens to be an Amazon wishlist. All that matters is that I can keep it in some service, somewhere, that can syndicate data to other services elsewhere.

The CTO Breakfast will be held this Thursday, March 27 at 8am in the Novell Cafeteria (Building G). Anyone interested in high-tech and product development is welcome. The discussion is free-form, so feel free to bring some topics to discuss.

Here is a list of upcoming meetings:

• Apr 17 (Thursday)
• May 30 (Friday)
• June 27 (Friday)

Please get them on your calendar!

Tags: utah events cto breakfast

Utah will hold caucus meetings tonight for the purpose of selecting delegates to the county and state conventions. Your voice is amplified many times over by being part of the process that decides who's on the ballot rather than just selecting from a few candidates in November.

The caucus meeting with be with people in your neighborhood--probably people you know. You can find out what precinct you live in by clicking here. Then you can visit the Republican or Democratic Web sites to determine where you should go. (If you're a member of a party other than these two, find your party Web site for caucus locations.)

Once you know where to go, just show up. Both the Republicans and Democrats are meetings at 7pm. You'll find it's a pretty friendly atmosphere. If you want to run as a delegate, it helps to take some friends with you who will vote for you. Being a delegate does take time and involve some meetings (not to mention the conventions themselves) but is a great way to further amplify your voice.

Personally, I'm planning to talk about Lessig's Change Congress movement (I wrote about it while I was at ETech) and my commitment to preferentially vote for candidates willing to take at least part of the Change Congress pledge. If enough delegates did this, we could at least change the part of Congress we're responsible for in Utah.

Tags: utah events politics

This week’s ITConversations show is a chat with Carl Malamud, whose exploits I’ve followed ever since he launched podcasting a decade ahead of schedule with a project called Internet Talk Radio. Since then, Carl’s mainly known for his tireless crusade to release troves of public information to the Net: SEC filings, patents, Congressional video, historical photographs, and most recently, U.S. case law.

One of the questions I wanted to explore with Carl is also raised here by John Montgomery:

Popfly, a mashup tool, depends on three things: data that is simple to access programmatically, interesting, and available under terms that enable users to work with it. As with most software endeavors, you can pick two.

The government has a huge amount of interesting data that’s available under really great terms. Weather? Check out http://www.noaa.gov. Financial information? Start with http://www.sec.gov. Crime statistics? Dig around in http://www.usdoj.gov/. But how much of this is programmatically accessible? Very little, as it turns out.

John mentions the Sunlight Foundation’s efforts to provide an intermediary layer of services that make raw data easier to access and manipulate, and I raised that point with Carl. From his perspective, of course, it all starts with the data which he is rightly focused on providing. Even though the U.S. is far ahead of many other countries in this regard, there are oceans of important information not yet available even in raw form.

Carl has enormous faith in the Net’s ability to interconnect and enhance these raw sources, and I do too. Here’s a small but significant example. If you view source on 28 Fed.R.Serv.3d 415, you’ll see one of my favorite strategies at work: semantic metadata encoded using CSS style tags. That enables an important kind of programmatic access. Now it’s true that today, Internet search engines don’t support queries that ask for documents where Shelby Reed appears as a plaintiff in an appeal to the U.S. Court of Appeals, Fifth Circuit. Someday, though, that kind query will be supported, and the latent semantics of this rendering of U.S. case law will emerge.

These enhanced services don’t necessarily just arise from the grassroots, however. Resource-rich organizations are often in the best position to provide them. One example, we agreed, is the New York Times’ stunningly effective visualization of presidential election debates. Ideally we’d be able to visualize all of the proceedings of Congress in the same way. That’s probably too much to expect of public-interest groups running shoestring operations. But what such groups can do is apply Carl’s favorite technique: Create a few high-profile examples, and then pressure the government into internalizing the process.

Lately, I've had a very rocky relationship with my Mac Book Pro. One of the things that attracted me to OS X was its stability. Over the past several months (before and after Leopard) my MBP has had trouble with sleeping, waking, and weird, inexplicable freezing. Often when the machine woke up, it would the screen would be black and never come back. The machine would freeze at odd times and nothing would unstick it. I couldn't even log in remotely using SSH, so it was pretty stuck.

The final straw was erratic mouse behavior. The mouse seemed sluggish and wouldn't follow the track. Only a reboot would cure it and the it would deteriorate over the next 5-10 hours.

I considered an OS reload, but didn't really expect that would solve the problem since these issues had persisted through reloads before. I suspected, but didn't have much evidence that it has something to do with a kernel extension because the locking up was occurring at a deep level.

The good news is that OS X some new tools for exploring what kernel extensions are loaded. I used the following command to see what (besides Apple extensions) were loaded:

kextfind -loaded -not -bundle-id -substring 'com.apple' -print 

Doing so revealed about five extensions. I started Google each one and discovered that vmmain.kext was suspected in at least one other case of causing erratic mouse behavior. I didn't want to uninstall Parallels to test this, so I just renamed the plist file in StartupItems so it wouldn't load.

mv /Library/StartupItems/Parallels/StartupParameters.plist foo.plist

Now, after a reboot, Parallels doesn't load and looking at the loaded kernel extensions shows that in particular vmmain.kext hasn't loaded.

I did this five days ago and my machine has been remarkably stable. It feels like my old Mac again. I don't know that it's a Parallels problem--at least not exclusively. I suspect that its an interaction with other things. In particular, I run Parallels and Fusion both and there may be some weird interaction going on there.

I like Parallels. I like Coherence better than Unity. I like the snapshot feature in Parallels because it allows multiple snapshots of the same image. But I need Fusion for running Fedora (Parallels didn't work so well for me there). Fusion also wins on the performance front--particularly with multiple cores.

There are ways to load and unload kernel extensions and that may be a better solution, but for now, I'm just using Fusion to see what happens. I'll let you know if my experiment turns up anything else.

Tags: osx parallels virtualization debugging fusion vmware

The second installment of Perspectives is up, with Vittorio Bertocci, author of Understanding Windows CardSpace. This interview was recorded a few months ago, and has been waiting for the Perspectives site to launch. In January I excerpted the part about omnidirectional identity, a difficult phrase that I continue to struggle with. Maybe a better one is Internet persona: the social mask that you project when you self-publish online, and to which reputation attaches. Whatever we call this phenomenon, its Laws of Identity — not only for people, but also for digital object — are not yet well defined.

Most of the interview, though, concerns the existing “unidirectional” mechanisms supported by CardSpace. I asked Vittorio to relate those mechanisms to precursors like SSL client certificates and Kerberos, and also to the complementary OpenID system. As discussed in my ITConversations podcast with Dick Hardt, the principles that govern this identity machinery are abstract and, until we experience them firsthand, will be hard for most of us to grasp. But Vittorio does a good job of explaining those principles in terms of concrete examples.

While reviewing a white paper by a colleague on the subject of personal digital archives, I realized that I hadn’t followed through on a plan to consolidate a few different caches of digital photos from various digicam and computer eras. So of course, when I went looking, things weren’t exactly the way I remembered. One particular batch was missing, and there were some anxious moments while I booted up dormant computers and mounted shelved disks. In the end I found the missing set, but although I could have sworn they were in three safe places, there was really only one.

In these moments of panic, the need for a lifebits service becomes crystal clear. But the moments pass, and we move on. Most people, most of the time, don’t yet feel the need for that kind of service.

Inevitably that will change. I wonder how, and when?

I thoroughly enjoyed Jon Udell's interview with Ward Cunningham on IT Conversations. They talk a lot about Ward's efforts at the Eclipse Foundation to build transparent workflow systems. That is, as Jon puts it:

But what if you could find out, before pressing the Save button, what's going on in that black box? And what if your way of finding out wasn't by reading bogus documentation, but instead by probing the system itself using its own test framework?

From Ward Cunningham's Visible Workings « Jon Udell
Referenced Thu Mar 20 2008 08:42:43 GMT-0600 (MDT)

You'll want to read Jon's description of Ward's visible workings along with the podcast to get the most out of it. Better yet, I'd love to have a screencast of the system at work.

At one point Jon and Ward talk about how this might apply to eGovernment. Think about a button you could push at any point that would tell you how your current interaction with a government Web site was likely to proceed. I'd love to see it.

Tags: testing programming workflow itconversations egovernment

I’m running a couple of services that make automatic use of Amazon wishlists, and today I noticed that the current version of the API is going away:

503 - Service Unavailable

ECS3 is currently unavailable due to a planned outage in preparation for the complete shutdown of ECS3 on March 31, 2008.

After March 31, 2008, we will no longer accept Amazon ECS 3.0 requests. Please upgrade to the Amazon Associates Web Service (previously called Amazon E-Commerce Web Service 4.0) by then to ensure that you or your customers are not affected by the upcoming deprecation.

Amazon ECS 3.0 deprecation was announced a year ago in February 2007. You can read the original post at http://developer.amazonwebservices.com/connect/ann.jspa?annID=164.

In preparation of the March 31st deprecation, the Amazon ECS 3.0 web service will experience several outages. The complete outage schedule can be viewed at http://developer.amazonwebservices.com/connect/ann.jspa?annID=276.

Please refer to the migration guide for assistance in mapping Amazon ECS 3.0 calls to their Amazon Associates Web Service 4.0 equivalents. You can find the migration guide at http://developer.amazonwebservices.com/connect/entry.jspa?categoryID=12&externalID=627. Please use the Amazon Associates Web Service forum to ask technical questions and share answers with your fellow developers.

We thank you for being part of Amazon’s Developer community and look forward to your continued support.

Like Rich Burridge, I’ll be needing a replacement for PyAmazon, the Python module Mark Pilgrim wrote long ago to simplify use of the original Amazon API.

In our modern world of aggregation, search, and syndication, it’s easy to wait and see what will happen. I went to bloglines and searched for blog items that — like Rich’s and now mine — point to Amazon’s page about migrating to the new API. And then I subscribed to that search.

In a way, this is too easy. I can imagine a bunch of people camped on that query, watching the clock and waiting for someone else to step up to the plate before March 31. The first time around, when Amazon web services were new and shiny, it was cool to be that person. Now, not so much.

Update: A couple of folks have pointed to PyAWS. As mentioned in Rich Burridge’s blog entry, it doesn’t seem to offer, e.g., a single call to retrieve all items from a wishlist. However, when I reviewed my use of the earlier PyAmazon, in terms of raw interaction with the RESTful API and its XML output, I remembered how simple that interaction was. It’s just as simple in the new Amazon API, just slightly different. Encapsulating what I needed to do required only a few lines of code.

Generalizing that encapsulation is much harder. And when you have to repeat that hard work for many different languages, and for many different APIs, the inevitable result is that these per-language API wrappers tend to lag.

That’s one reason I’m looking forward to services built on Astoria ADO.NET Data Services, or an equivalent normalization layer. I think it can substantially narrow the gap between RESTful APIs and the convenience wrappers we enjoy in various programming languages.

When Americans (led by Lou Dobbs) scream "do something about illegal immigration" at the top of their lungs, you know we're going to get saddled with a bunch of awful ideas. One of those is Electronic Employment Verification, or EEV. EEV is a plan to create a big federal database of everyone eligible to work. Before a potential employer could give you a job, you'd have to be "cleared to work" by the Feds. Even worse, it will just be an API call to a big database in the sky.

In theory, this seems like a great solution. After all, if we just had a nice API call to make to determine whether or not to give jobs to people we could remove the magnet of lucrative jobs, right? In theory. The problem is the world's a messy place.

in the pilot program that DHS has been running, the error rate is 4.1%. If you do that math (55 million new hires each year in the US), that works out to 11,000 people mistakenly classified as ineligible to work every day! Of course, that won't be you, right? Of course, the error rate is bound to get better, right? Of course, there will be a swift and fair adjudication process, right? If you believe that you don't understand IT or the government very well.

What's worse, it won't even solve the immigration program. It will simply make the value of your identity data all the greater. If you make something worth more, it's more likely to be stolen. Rather than solving the illegal immigration problem EEV will more likely simply increase the incidence of identity theft. Your government at work. Rather than making you safer from identity theft, they're going to make you more susceptible.

Even if you're not one to go in for Orwellian scares about Big Brother growing out of a national ID system, this one has to make you wonder what people in Washington are thinking.

What can you do? Call you congressperson tomorrow and tell them EEV is a bad idea. Give your credentials. Make sure they understand people in their district will have a harder time getting employment. Increasing a tiny bit of friction in the job process is likely to cause big costs for business and government.

For more information and details on EEV, read Jim Harper's excellent policy analysis on it. Disclosure: he quotes me.

Tags: politics identity

This week on ITConversations I have a two-part interview with Ward Cunningham. In part one, we explore his implementation of Brian Marick’s visible workings idea, which combines software testing with business process transparency. This is one of those transformative ideas that will not, at first, seem interesting and important to most people. And maybe it never will. But then again, Ward has a track record. The wiki idea didn’t at first seem interesting and important to most people either, and look what’s happened there. So, you never know. Maybe in 2020 we’ll notice that business software is a lot more reliable and understandable than it used to be, and we’ll look back and say: Ward did it again.

In part two, we discuss Ward’s new wiki-based venture, aboutus.org. It’s a directory that aims to become a sort of extended WHOIS database, where domain name owners — along with anyone who reads the websites attached to those domains — can collaboratively describe the people, companies, and organizations represented by those websites. I like the concept, but I wish it weren’t necessary to sign up in order to update http://aboutus.org/jonudell.net. Instead I’d prefer to describe myself on my own hosted lifebits service, wherever that might be, and then syndicate the information to aboutus.org and elsewhere.

I just finished reading Dreams from My Father: A Story of Race and Inheritance over the weekend. Like many I'm intrigued with Barack Obama and would like to understand him better. I feel like reading the book certainly helped in that quest, but I can't say that it made more--or less--inclined to vote for him.

First, ignoring politics, this is a great book. I enjoyed it thoroughly, sometimes forgetting that what I was reading was autobiography because the story was so good it felt like a novel. Here's a sample from the book (pg 327), one of many I flagged:

What is a family? Is it just a genetic chain, parents and offspring, people like me? Or is it a social construct, an economic unit, optimal for child rearing and divisions of labor? Or is it something else entirely: a store of shared memories, say? An ambit of love? A reach across the void?

I could list various possibilities. But I'd never arrive at a definite answer, aware early on that, given my circumstances, such an effort was bound to fail. Instead, I drew a series of circles around myself, with borders that shifted as time passed and faces changed but that nevertheless offered the illusion of control. An inner circle, where love was constant and claims unquestioned. Then a second circle, a realm of negotiate love, commitments freely chosen. An then a circle for colleagues, acquaintances; the cheerful gray-haired lady who rag up my groceries back in Chicago. Until the circle finally widened to embrace a nation or race, or a particular moral course, and the commitments were no longer tied to face or a name but were actually commitments I'd made to myself.

This is the story of a man of mixed parentage finding himself and where he belongs. The book ends recounting experiences that Obama had visiting his family in Africa--a large extended group that seem, for the most part, to have welcomed him and given him a sense of who he was for maybe the first time.

Now to the political. Obama speaks frequently of an America that transcends race. I think that is much of his appeal--for voters of all races. And yet, if you read the book looking for evidence that he's transcended race himself, you'll find very little. The book is very much about race and his search for his own identity in America in the 70's and 80's.

That's not bad--it's who he was. I'm simply saying that Obama's story is not one of a child of a white woman and black man bringing the divided races together. Sometimes it feels like the opposite as he recounts his experiences.

There's a certain element of redemption in the book. If you look at Obama's trajectory when he was in high school, you wouldn't have picked him to go to Harvard Law and edit the Law Review there--let alone be President of the United States. And yet his story is familiar to anyone who has seen good kids lose their way and then return to the values they learned as children later in life.

And that, I think, is a nice allegory for the hold Barack Obama has over America: he promises redemption for America's past. The ambiguous "hope" and "change" promised by Obama is interpreted by many that we can get past the discomfort of race in American discourse and move on to a different, better place.

And yet, redemption isn't the entirety of Obama's appeal. I find myself drawn to his message not because of his race, but because of his age. I think a McCain-Obama race could shape up to be as much about generational differences as party differences. Many younger voters are going to resonate with Obama's world view and references because he's closer to their age. McCain just isn't that hip.

In the end, I decided that while I enjoyed the book immensely nd would certainly recommend it, it didn't tell me all I wanted to know about Barack Obama. I've picked up Audacity of Hope and will read it as soon as I finish No Country for Old Men (I can't put that one down).

P.S. Small world: I went to Google to find a picture of Obama for this post and found this entry from Eve Maler who went to high school with Barack--or Barry as he was known then in Hawaii. Complete with high school year book page. Check it out!

Tags: politics

I wasn’t going to post this humorous anecdote but Mike Caulfield reminded me that it’s too funny not to share. After musing about a subscription service for running shoes, I walked in my local store, bought a new pair, and invited them to notify me in three months. Hilarity ensued.

He: We’re not really set up to do that.

Me: You could email me.

He: Yeah, but then we’d have to keep some kind of customer database on the computer.

Oh, right. Having a database of customers who’ve invited you to contact them on a regular basis … that’d suck, wouldn’t it?