Thursday, January 03, 2008
by Nik Kalyani
Thursday, January 03, 2008 2:04:06 PM (Pacific Standard Time, UTC-08:00)
Lots of blog buzz about Facebook shutting-down Scoble's account for running a script against Facebook, thus violating the site's ToS. It appears that an ill-conceived experimental Plaxo Pulse script that used screen-scraping to retrieve email addresses is the culprit. I empathize with Scoble and given the facts, also think Facebook was justified in suspending his account. This post, however, is about a bigger, related issue that the event highlights. (No, it's not about who owns the data either.)

I am concerned about the general complacency and casual attitude that people generally have about Web 2.0 data security. In this case, an extremely tech savvy individual allowed a test script from another (supposedly) tech savvy company to be executed against production data. That's insane, no?

Not quite...it's no different than downloading a beta app from a website and allowing it to run on your desktop. You really have no idea how it's going to fudge-up your machine and your data, but you probably do it anyway once you determine the risk is acceptable.

In both situations, it comes down to Trust, Ignorance and Recklessness or a combination thereof. While there are now adequate protections available for desktop apps so your data can be reasonably protected, the same is not true for Web 2.0 apps. It's the Wild West out there. With the proliferation of API's, widgets and mash-ups, you have no idea where your data is being stored, who has access to it and what apps that are accessing it are doing with it. None. And you don't have a prayer of a chance of ever finding out.

We need standards not only for open, cross-site access to user data, but we also need standards that provide for how the data is persisted by different sites. I don't have the time to read the ToS for each site, and even if I did, I have no way of verifying that what they say about the privacy and security of my data is actually what is happening in the data center. With a high probability, I can assure you that it is seldom as air-tight as the ToS legalese would have you believe.

DataPortability.org appears to be a good start for enabling access to ones data, but that is only one half of the equation. In a distributed online world, we need standards that provide transparency about how the data is being stored, verifiable means for ensuring that ToS are being adhered to by sites and auditability about when/by whom/how our data is accessed. We need bread crumbs associated with user data no matter where it is persisted and this information needs to be accessible to us in an easy, centralized location regardless of where the data is stored. Better yet, we need a standard means of encrypting our data across multiple sites. This is a tall order, and it is unlikely to happen anytime soon, but it will happen. There will be a standard because users will demand it and sites that don't provide it will see their users leave en masse.

Today, it's not a big deal for most people because their online information consists of photos and videos, but the more important stuff is still in silos. Our bank has some info, the credit card company has some and the travel company has some. How long before these silos also start having API's? Wesabe is already doing it. Others will follow. Very soon, it's going to be way more personal information than just pictures of your kid at the family picnic that are going to be strewn across the web.

We either have to give up most or all of our privacy or figure out a way to protect it as the distributed web evolves.

 Thursday, December 08, 2005
by Nik Kalyani
Thursday, December 08, 2005 7:52:48 PM (Pacific Standard Time, UTC-08:00)

Umair could not have made a more timely post about Yahoo’s lack of innovation. He discusses Yahoo’s “hyperimitation” and how it never really improves anything in any meaningful way.

I say timely because I have now spent some time engineering a better “forums” application because I have given up on forum software available today. Many of us use web-based discussion forums everyday, and they are all the same. From the common seed of BBS many years ago, there evolved a design for web discussions — forum groups, forums, threads and posts. It all worked quite well until everybody had a computer, a browser, an Internet connection and an opinion. This has resulted in three major problems:

– It’s hard to keep up with the volume of information. With hundreds and sometimes thousands of posts in different forums, across different sites, how does one track information of interest and more importantly relevance. Email digests and email subscriptions don’t cut it. They just move the information to different client software. If you track even 3–5 busy forums, you will be innundated with information.

– It’s hard to find relevant information. Forums are notoriously hard to search because there is no meaningful way to perform contextual searches. Users freely post anything (even in moderated forums) and there is no guarantee that threads will stay focused on a topic. Keyword searches yield tons of results and out of frustration, users will often just post a question which has already been answered umpteen times.

– It’s hard to tell which user’s posts are worth reading. Just because a particular user has a high post count does not make them interesting, relevant or for that matter, credible. For instance, there are many users on the ASP.Net forums with huge post counts that have very little substance in their posts.

As some of you may have noticed, I have not been very active blogging or on the forums lately. And yes, I am also behind on releasing updates to some of Speerio products. This is because I have been hard at work doing a fair amount of re-architechting of existing products, while also working on a Speerio product suite called “Community Studio” (more on that next week). One of the apps in the suite is a discussion application. In working out the design for this app, I was tempted to take the interaction metaphor used by most most forums today and implement it, perhaps with some U.I. goodies to make things interesting. But the more I thought about it, the more I disliked this idea. There is no point in replicating a bad concept with a good U.I. — all it does is give you more of the same problems, albeit with better user interaction. A good example of this is the recent upgrades made to the ASP.net forums. Many client-side U.I. enhancements, but none that substantially help with usability and easily finding relevant information.

I decided that I would take a fresh approach to designing a web-based discussion application (I’m not going to call it a forum any more). Starting with a clean slate, I prioritized some key requirements —

1) The application should enable a user to instantly know what topics are discussed the most.

2) The application should enable a user to instantly find all discussions on a topic, prioritized by relevance, chronology or amount of participation for all time or a user-specified period of time.

3) The application should enable the user to be notified when more discussions about topics of interest to her/him emerge.

4) The application should provide the user with hints about contextually relevant topics that have been discussed when he/she searches for a topic.

5) The application should be incredibly easy to use, very responsive and should be optimized to take advantage of the one trait that is common to all web-based discussions — 99.99% of the time all content is write-once/read-many.

Looking at those requirements, it should be obvious that the group – forum – thread metaphor isn’t going to cut it. The more I thought about it, the more I became convinced that web-discussion applications have been over-engineered and the solution is not to add, but to subtract. Doing just that, I arrived at a design that is quite different from today’s discussion software, but which I think meets all the above requirements. I’ll go into detail in a later blog post, but let me share some high-level ideas here:

1) Information navigation is done using a folksonomy-generated tag cloud.

2) There are no groups or sub-groups or forums…only a flat collection of discussions (100 or 1,000,000…doesn’t matter)

3) Each discussion is natively stored as an RSS feed. A discussion is the originating post and all responses to that post. There is no hierarchical thread, just a linear series of responses.

4) Each user who posts a response, must tag her/his response with tags that are relevant to the content of her/his post. This is key because contextual relevancy is best determined by humans and not computers (at least given today’s technology).

5) Tag clusters are automatically created for contextual relevancy.

This, in a nutshell, is a design that meets all the requirements. Of course, there is a lot of detail in the implementation and the U.I., but from a 20,000–foot perspective, the design is very, very simple. While technical gurus are busy arguing the merits of a folksonomy over structured taxonomies, ordinary people are enjoying their ease of use, both for content creation and retrieval. This makes me confident that even though my folksonomy-based discussion app will take some getting used to, once users see how it makes web-based discussions easier to track, digest and retrieve, they will love it.

More design details and information on Community Studio to follow.

 Sunday, August 07, 2005
by Nik Kalyani
Sunday, August 07, 2005 10:17:34 PM (Pacific Standard Time, UTC-08:00)
 Wednesday, September 01, 2004
by Nik Kalyani
Wednesday, September 01, 2004 7:26:37 PM (Pacific Standard Time, UTC-08:00)
Eric Sink has an excellent series of articles on MSDN called The Business of Software.
RSS feed
Search and Links
Bling

View Nik Kalyani's profile on LinkedIn

TechBubble
www.flickr.com
This is a Flickr badge showing public photos from techbubble. Make your own badge here.
Statistics
Total Posts: 216
This Year: 19
This Month: 0
This Week: 0
Comments: 226
About the author/Disclaimer

Disclaimer
The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2008
Nik Kalyani
Sign In
All Content © 2008, Nik Kalyani