vanderwal.net

The Data Journalism Handbook is Available

The Data Journalism Handbook is finally available online and soon as the book Data Journalism Handbook - from Amazon or The Data Journalism Handbook - from O’Reilly, which is quite exciting. Why you ask?

In the October of 2010 the Guardian in the UK posted a Data Journalism How To Guide that was fantastic. This was a great resource not only for data journalists, but for anybody who has interest in finding, gathering, assessing, and doing something with the data that is shared found in the world around us. These skill are not the sort of thing that many of us grew up with nor learned in school, nor are taught in most schools today (that is another giant problem). This tutorial taught me a few things that have been of great benefit and filled in gaps I had in my tool bag that was still mostly rusty and built using the tool set I picked up in the mid-90s in grad school in public policy analysis.

In the Fall of 2011 at MozFest in London many data journalist and others of like mind got together to share their knowledge. Out of this gathering was the realization and starting point for the handbook. Journalists are not typically those who have the deep data skills, but if they can learn (not a huge mound to climb) and have it made sensible and relatively easy in bite sized chunk the journalists will be better off.

All of us can benefit from this book in our own hands. Getting to the basics of how gather and think through data and the questions around it, all the way through how to graphically display that data is incredibly beneficial. I know many people who have contributed to this Handbook and think the world of their contributions. Skimming through the version that is one the web I can quickly see this is going to be an essential reference for all, not just journalists, nor bloggers, but for everybody. This could and likely should be the book used in classes in high schools and university information essentials taught first semester first year.

Trip and d.construct Wrap-up

I am back home from the d.construct trip, which included London and Brighton. The trip was very enjoyable, the d.construct conference is a pure winner, and I met fantastic people that keep my passion for the web alive.

d.construct

The d.construct conference had Jeff Barr from Amazon talking about Amazon Web Services, Paul Hammond and Simon Willison discussing Yahoo and its creation and use of web services for internal and external uses, Jeremy Keith discussing the Joy of the API, Aral Balkan presenting the use of Adobe Flex for web services, Derek Featherstone discussing accessibility for Javascript and Ajax and how they can hurt and help the web for those with disabilities, myself (Thomas) discussing tagging that works, and Jeff Veen pulling the day together with designing the complete user experience.

Jeff Barr provided not only a good overview of the Amazon offerings for developers, but his presentation kept me interested (the previous 2 times my mind wandered) and I got some new things out of it (like the S3 Organizer extension for Firefox.

Jeremy was his usual great presenting form (unfortunately a call from home caused me to miss the some of the middle, but he kept things going well and I heard after that many people learned something from the session, which they thought they knew it all already.

Paul and Simon did a wonderful tag team approach on what Yahoo is up to and how they "eat their own dog food" and how the Yahoo Local uses microformats (Wahoo!).

Aral was somebody I did not know before d.construct, but I really enjoyed getting to know him as well as his high energy presentation style and mastery of the content that showed Flash/Flex 2.0 are fluent in Web 2.0 rich interfaces for web services.

Derek was fantastic as he took a dry subject (accessibility) and brought it life, he also made me miss the world of accessibility by talking about how JavaScript and Ajax can actually improve the accessibility of a site (if the developer knows what they are doing - this is not an easy area to tread) and made it logical and relatively easy to grasp.

I can not comment on my own presentation, other than the many people what sought me out to express appreciation, and to ask questions (many questions about spamming, which is difficult if the tagging system is built well). I was also asked if I had somebody explain the term dogging (forgetting there was a rather bawdy use of the term in British culture and using the term as those people who are dog lovers - this lead to very heavy laughter). Given the odd technical problems at the beginning of the presentation (mouse not clicking) things went alright about 5 minutes or so in.

Lastly, the man I never want to follow when giving a presentation, Jeff Veen rocked the house with his easy style and lively interaction with his slides.

I am really wanting to hear much more from Aral and Derek now that I have heard them speak. I am looking forward to seeing their slides up and their podcasts, both should be posted on the d.construct schedule page.

London Stays

The trip also included an overnight stay in London on the front and back end of the conference. Through an on-line resource I had two last minute rooms booked at Best Western Premiers that were great rooms in well appointed hotels. The hotels even had free WiFi (yes, free in Europe is a huge savings), which was my main reason for staying at these hotels I knew nothing about. I really like both locations, one near Earls Court Tube Station and the other Charing Cross Road and SoHo. The rooms were well under 200 U.S. dollars, which is a rarity in central London. I think I have a new place to track down then next time I visit London.

London People & Places

I had a few impromptu meetings in London and an accidental chat. When I first got in I was able to clean-up and go meet friends Tom and Simon for lunch at China Experience. We had good conversations about the state of many things web. Then Tom showed me Cyber Candy, which I have been following online. I was then off to Neal's Yard Dairy to pick-up some Stinking Bishop (quite excellent), Oggleshield, and Berkswell. I then did a pilgrimage to Muji to stock up on pens and all the while using Yahoo Messanger in a mobile browser (a very painful way to communicate, as there is no alert for return messages and when moving the web connection seems to need resetting often).

That evening I met up with Eric Miraglia for a great chat and dinner, then included Christian Heillmann (who has a great new book (from my initial read) on Beginning JavaSctipt with DOM Scripting and Ajax) in our evening. The discussions were wonderful and it was a really good way to find people of similar minds and interests.

On my last day in London I ended up running into Ben Hammersley as he was waiting for a dinner meeting. It was great to meet Ben in person and have a good brief chat. Somehow when walking down the street and seeing a man in a black utilikilt, with short hair, and intently using his mobile there are a short list of possibilities who this may be.

Food

My trip I had a few full English breakfasts, including one in Brighton at 3:30am (using the term gut buster), which was my first full meal of the day. The breakfast at the Blanche House (the name of the hotel never stuck in my head and the keys just had their logo on them, so getting back to the hotel was a wee bit more challenging than normal) was quite good, particularly the scrambled eggs wrapped in smoked Scottish salmon. The food the first night in Brighton at the Seven Dials was fantastic and a great treat. Sunday brunch at SOHo Social in Brighton was quite good and needed to bring me back from another late night chatting, but the fish cakes were outstanding. The last evening in London I stopped in at Hamburger Union for a really good burger with rashers bacon. The burgers are made with only natural fed, grass-reared additive free beef. This is not only eco-friendly, but really tasty. I wish there were a Hamburger Union near where I work as I would make use of it regularly.

Too Short a Visit

As it is with nearly every trip this year, the time was too short and the people I met were fantastic. I really met some interesting and bright people while in Brighton and I really look forward to keeping in touch as well as seeing them again.

Cultures of Simplicity and Information Structures

Two Conferences Draw Focus

I am now getting back to responding to e-mail sent in the last two or three weeks and digging through my to do list. As time wears I am still rather impressed with both XTech and the Microlearning conferences. Both have a focus on information and data that mirrors my approaches from years ago and are the foundation for how I view all information and services. Both rely on well structured data. This is why I pay attention and keep involved in the information architecture community. Well structured data is the foundation of what falls into the description of web 2.0. All of our tools for open data reuse demands that the underlying data is structured well.

Simplicity of the Complex

One theme that continually bubbled up at Microlearning was simplicity. Peter A. Bruck in his opening remarks at Microlearning focussed on simplicity being the means to take the complex and make it understandable. There are many things in the world that are complex and seemingly difficult to understand, but many of the complex systems are made up of simple steps and simple to understand concepts that are strung together to build complex systems and complex ideas. Every time I think of breaking down the complex into the simple components I think of Instructables, which allows people to build step-by-step instructions for anything, but they make each of the steps as reusable objects for other instructions. The Instructables approach is utterly brilliant and dead in-line with the microlearning approach to breaking down learning components into simple lessons that can be used and reused across devices, based on the person wanting or needing the instruction and providing it in the delivery media that matches their context (mobile, desktop, laptop, tv, etc.).

Simple Clear Structures

This structuring of information ties back into the frameworks for syndication of content and well structured data and information. People have various uses and reuses for information, data, and media in their lives. This is the focus on the Personal InfoCloud. This is the foundation for information architecture, addressable information that can be easily found. But, in our world of information floods and information pollution due to there being too much information to sort through, findability of information is important as refindability (this is rarely addressed). But, along with refindability is the means to aggregate the information in interfaces that make sense of the information, data, and media so to provide clarity and simplicity of understanding.

Europe Thing Again

Another perspective of the two conferences was they were both in Europe. This is not a trivial variable. At XTech there were a few other Americans, but at Microlearning I was the only one from the United States and there were a couple Canadians. This European approach to understanding and building is slightly different from the approach in the USA. In the USA there is a lot of building and then learning and understanding, where as in Europe there seems to be much more effort in understanding and then building. The results are somewhat different and the professional nature of European products out of the gate where things work is different than in the USA. This was really apparent with System One, which is an incredible product. System One has all the web 2.0 buzzwords under the hood, but they focus on a simple to use tool that pulls together the best of the new components, but only where it makes sense to create a simple tool that addresses complex problems.

Culture of Understanding Complex to Make Simple

It seems the European approach is to understand and embrace the complex and make it simple through deep understanding of how things are built. It is very similar to Instructables as a culture. The approach in the USA seems to include the tools, but have lacked the understanding of the underlying components and in turn have left out elements that really embrace simplicity. Google is a perfect example of this approach. They talk simplicity, but nearly every tool is missing elements that make it fully usable (calendar not having sync, not being able to only have one or two Google tools on rather than everything on). This simplicity is well understood by the designers and they have wonderful solutions to the problems, but the corporate culture of churning things out gets in the way.

Breaking It Down for Use and Reuse

Information in simple forms that can be aggregated and viewed as people need in their lives is essential to us moving forward and taking the pain out of technology that most regular people experience on a daily basis. It is our jobs to understand the underlying complexity, create simple usable and reusable structures for that data and information, and allow simple solutions that are robust to be built around that simplicity.

More XTech 2006

I have had a little time to sit back and think about XTech I am quite impressed with the conference. The caliber of presenter and the quality of their presentations was some of the best of any I have been to in a while. The presentations got beneath the surface level of the subjects and provided insight that I had not run across elsewhere.

The conference focus on browser, open data (XML), and high level presentations was a great mix. There was much cross-over in the presentations and once I got the hang that this was not a conference of stuff I already knew (or presented at a level that is more introductory), but things I wanted to dig deeper into. I began to realize late into the conference (or after in many cases) that the people presenting were people whose writting and contributions I had followed regularly when I was doing deep development (not managing web development) of web applications. I changed my focus last Fall to get back to developing innovative applications, working on projects that are built around open data, and that filled some of the many gaps in the Personal InfoCloud (I also left to write, but that did get side tracked).

As I mentioned before, XTech had the right amount of geek mindset in the presentations. The one that really brought this to the forefront of my mind was on XForms, an Alternative to Ajax by Erik Bruchez. It focussed on using XForms as a means to interact with structured data with Ajax.

Once it dawned on me that this conference was rather killer and I sould be paying attention to the content and not just those in the floating island of friends the event was nearly two-thirds the way through. This huge mistake on my part was the busy nature of things that lead up to XTech, as well as not getting there a day or two earlier to adjust to the time, and attend the pre-conference sessions and tutorials on Ajax.

I was thrilled ot see the Platial presentation and meet the makers of the service. When I went to attend Simon Willison's presentation rather than attending the GeoRSS session, I realized there was much good content at XTech and it is now one on my must attend list.

As the conference was progressing I was thinking of all of the people that would have really benefitted and enjoyed XTech as well. A conference about open data and systems to build applications with that meet real people's needs is essential for most developers working out on the live web these days.

If XTech sounded good this year in Amsterdam, you may want to note that it will be in Paris next year.

Light Overview of XTech and Amsterdam (including BarCamp Amsterdam)

This trip to Amsterdam for XTech 2006 (and now bits of BarCamp Amsterdam II has been quite different from previous trips, in that Amsterdam is now getting to be very familiar. I also did not spend a day on the front end of the trip walking around adjusting to the time change and spent it inside at XTech, where I saw many friends, which really made it feel more like a floating island comprised of geographically distributed friends that I see when I travel.

It has been great seeing good friends that I really wish I could see more and/or work with on projects as I believe some killer things could get done. I also met people and got to hang out with many new people, which is always great. I was pleased to spend time with people I have only partially spent time with in the past.

I quite enjoyed XTech as it was a good amount of geekery, which provided sparks of inspiration, very good feedback on the "Come to Me Web" and "Personal InfoCloud" stuff I presented. The session had Paul Hammond, Tom Coates, and then myself presenting ideas that focussed on open data, using open data, and building for personal use and reuse of information in our three presentations. It was a fantastic set up.

There were many Mozilla folks around, which was fantastic to hear where the Mozilla/Firefox development is going. This was a very good cross pollination of people, ideas, and interests.

I also realized I need to through out my presentation on Personal InfoCloud and Come to Me Web and rebuild it from scratch. I was finding that my presentation that I have been iterating on for the past year or so is something that needs restructuring and refocussing. I get very positive comments on the presentation, but in delivering the presentation I have made many minor tweaks that have disrupted my flow of delivery. I believe that starting from scratch will help me focus on what gets delivered when. I am really do not write out the presentation in long form as I think that would make it stale for me.

I am heading home tomorrow, but I have not quite felt like I was in Amsterdam as it is really no longer a foreign place. It is still one of my favorite places to be. I spent much time exploring thoughts, spending time with people, playing with digital things, but not deeply finding the new bits of Amsterdam (outside of a few hours this morning). Ah well, I am back in a few short weeks.

Live Data Could Solve the Social Bookmarking Problem with Information Volatility

Alex brings up something in his Go and microformat stuff! covering what is in the works with Microformats at Microsoft. Scroll down to where Alex talks about "mRc = Live data wiring", now this live data access is incredibly important.

One of the elements that has been bugging me with social bookmarking it the volatility of the information is not taken into account when the bookmark is made. No, I am not talking about the information blowing up, but the blood pressure of the person bookmarking may rise if the data changes in some way. I look at social bookmarking, or bookmarking in general as a means to mark a place, but it fails as an indicator of state or status change of the information we are pointing to. The expressing of bookmarking and/or tagging is an expression of our explicit interest in that object we bookmarked and/or tagged. The problem is our systems so far are saying, "yes, you have interest, but so what".

What the live data approach does is makes our Personal InfoCloud active. If we could bookmark information and/or tag chunks of information as important we should be able to find out when that information changes, or better get an alert before the information changes. One area where this is essential and will add huge value is shopping. What happens with products in the real world? The prices change, they go out of stock, the product is modified, production of the product stopped, etc. The permeations are many, but those expressing interest should be alerted and have their information updated.

One of the things I have been including in my "Come to Me Web" presentations is the ability to think about what a person needs when they use and want to reuse information. We read about a product we desire, we read the price, but we may think about the product or put it on a wish list that is related to an event in the future. When we go to act on the purchase the information we have gathered and bookmarked may be out of date.

One solution I have been talking about in my presentations is providing an RSS/ATOM feed for the page as it is bookmarked so the person gets the ability to get updated information. I have built similar functionality into past products years ago that let people using data know when the data changed (e-mail) but also provided the means to show what the data was prior and what it had changed to. It was functionality that was deeply helpful to the users of the system. Live data seems a more elegant solution, if it provides the means to see what information had changed should the person relying on or desiring the information want it.

For Many AJAX is Not Degrading, But it Must

A little over two months ago Chad Dickerson posted one of the most insightful things on his site, Web 0.1 head-to-head: 37Signals' Backpackit vs. Gmail in Lynx. You are saying Lynx? Yes! The point is what 37Signals turns out degrades wonderfully and it is still usable. It could work on your mobile device or on a six year old low end computer in Eritrea in a coffee house or internet cafe (I have known two people who have just done that in the last year and found Gmail did not work nor did MSN, but Yahoo did beautifully).

Degrading is a Good Thing

Part of my problem with much of the push towards AJAX (it is a good, no great thing that XMLHTTPRequest is finally catching on). But, it must degrade well. It must still be accessible. It must be usable. If not, it is a cool useless piece of rubbish for some or many people. I have been living through this with airline sites (Continental), commerce sites (Amazon - now slightly improved), actually you name it and they adopted some where in this past year. In most cases it did not work in all browsers (many times only in my browser of last resort, which by that time I am completely peeved).

When Amazon had its wish list break on my mobile device (I (and I have found a relatively large amount of others this past couple years doing the same thing) use it to remember what books I want when I am in brick bookstores and I will check book prices as well as often add books to my wish list directly) I went nuts. The page had a ghastly sized JavaScript, which did some nice things on desktops and laptops but made the page far too large to download on a mobile device (well over 250 kb). In the past few weeks things seemed to have reversed themselves as the page degrades much better.

Is There Hope?

Chad's write-up was a nice place to start pointing, as well as pointing out the millions of dollars lost over the course of time (Continental admitted they had a problem and had waived the additional phone booking fee as well as said their calls were up considerably since the web redesign that broke things for many). Besides Chad and 37Signals I have found Donna Mauer's Designing usable rich internet applications as a starting point. I also finally picked up DOM Scripting: Web Design with JavaScript and the Document Object Model by Jeremy Keith, which focusses on getting JavaScript (and that means AJAX too) to degrade. It is a great book for designers, developers, and those managing these people.

I have an awful lot of hope, but it pains me as most of us learned these lessons five to seven years ago. Things are much better now with web standards in browsers, but one last hurdle is DOM standardization and that deeply impacts JavaScript/DOMScripting.

Structured Blogging has (Re)Launched

Structured Blogging has launched and it may be one of the brightest ideas of 2005. This has the capability to pull web services into nearly every page and to aggregate information more seamlessly across the web. The semantic components help pull all of this together so services can be built around them.

This fits wonderfully in the Model of Attraction framework by allowing people and tools to attract the information they want, in this case from all around the web far more easily than ever before.

[Update] A heads-up from Ryan pointed out this is a relaunch. Indeed, Structured Blogging is pointing out all of the groups that are supporting and integrating the effort. The newest version is of Structured Blogging is now microformat friendly (insanely important).

Microformats hCard and hCalendar Used for Web 2.0 Conference Speakers

Tantek has posted new microformat favelets (bookmarklets you put in your browser's toolbar). The microformat favelets available are: Copy hCards; Copy hCalendars; Subscribe to hCalendars; feed Copy hCalendars (beta); Subscribe to hCalendars feed (beta). Look at Tantek's Web 2.0 Speakers hCard and hCalendar blog post to understand the power behind this.

Microformats are one of the ways that sites can make their information more usable and reusable to people who have an interest. If you have a store and are providing the address you have a few options to make it easy for people, but a simple option seems to be using the microformat hCard (other options include vCard and links to the common mapping programs with "driving directions").

There will be more to come on microformats in the near future here.

Minor Changes in Off the Top

Last night I was able to add back the Quick Links (my current bookmarks from del.icio.us. This was due in great part to the folks at del.icio.us who now have a JavaScript that makes the process easy on you and easy on them (I am not sure how accessible this is as I have not tested it, but normally they are not accessible).

I also brought back to the link to just the Off the Top RSS feed, which has nothing but the last 10 entries in archaic RSS .91 format. I still am offering the wonderful Feedburner for Off the Top option, which has Off the Top entries, my del.icio.us entries, and my Flickr photo feed all bundled in one. I have quite a few people reading this in RSS on mobile devices at the moment and I thought I would make it easier for other that are going that route to get just the content of Off the Top.

Replacement RSS and XML Button

Mike just posted a killer international and language-free RSS logo button on his site. I really like it. Mainly is works for those of use who understand the RSS text version, but for those who are not as technically forward or in non-English/Western languages this could still work. The RSS and XML text on the buttons always need explanation to those not familiar with the terms. The end of many of the tutorials is often, "just click it, you do not really need to know what it means, just click". Something tells me Mike is on to something profound yet so wonderfully simple.

Response to Usability of Feeds

Jeffrey Veen has a wonderful post about the usability of RSS/Atom/feeds on his site. I posted a response that I really want to keep track of here, so it follows...

I think Tom's pointer to the BBC is a fairly good transition to where we are heading. It will take the desktop OS or browser to make it easier. Neither of these are very innovative or quickly adaptive on the Windows side of the world.

Firefox was the first browser (at least that I know of) to handle RSS outside the browser window, but it was still done handled in a side-window of the browser. Safari has taken this to the next step, which is to use a mime-type to connect the RSS feed to the desktop device of preference. But, we are still not where we should be, which is to click on the RSS button on a web page and dump that link into ones preferred reader, which may be an application on the desktop or a web/internet based solution such as Bloglines.

All of this depends on who we test as users. Many times as developers we test in the communities that surround us, which is a skewed sample of the population. If one is in the Bay Area it may be best to go out to Stockton, Modesto, Fresno, or up to the foothills to get a sample of the population that is representative of those less technically adept, who will have very different usage patterns from those we normally test.

When we test with these lesser adept populations it is the one-click solutions that make the most sense. Reading a pop-up takes them beyond their comfort zone or capability. Many have really borked things on their devices/machines by trying to follow directions (be they well or poorly written). Most only trust easy solutions. Many do not update their OS as it is beyond their trust or understanding.

When trends start happening out in the suburbs, exurbs, and beyond the centers of technical adeptness (often major cities) that is when they have tipped. Most often they tip because the solutions are easy and integrated to their technical environment. Take the Apple iPod, it tipped because it is so easy to set up and use. Granted the lack of reading is, at least, an American problem (Japanese are known to sit down with their manuals and read them cover to cover before using their device).

We will get to the point of ease of use for RSS and other feeds in America, but it will take more than just a text pop-up to get us there.

State is the Web

The use and apparent mis-use of state on the web has bugged me for some time, but now that AJAX, or whatever one wants to call "XMLHttpRequests", is opening the door to non-Flash developers to ignore state. The latest Adaptive Path essay, It's A Whole New Internet, quotes Michael Buffington, "The idea of the webpage itself is nearing its useful end. With the way Ajax allows you to build nearly stateless applications that happen to be web accessible, everything changes." And states, "Where will our bookmarks go when the idea of the 'webpage' becomes obsolete?"

I agree with much of the article, but these statements are wholly naive in my perspective. Not are they naive, but they hold up examples of the web going in the wrong direction. Yes, the web has the ability to build application that are more seemless thanks to the that vast majority of people using web browsers that can support these dynamic HTML techniques (the techniques are nothing new, in fact on intranets many of us were employing them four or five years ago in single browser environments).

That is not the web for many, as the web has been moving toward adding more granular information chunks that can be served up and are addressible. RESTful interfaces and "share this page" links are solutions. The better developers in the Flash community has been working to build state into their Flash presentations to people can link to information that is important, rather than instructing others to click through a series of buttons or wait through a few movies to get to desired/needed information. The day of one stateless interface for all information was behind us, I hope to hell it is not enticing a whole new generation of web developers to lack understanding of state.

Who are providing best examples? Flickr and Google Maps are two that jump to mind. Flickr does one of the best jobs with fluid interfaces, while keeping links to state that is important (the object that the information surrounds, in this case a photograph). Google Maps are stunning in their fluidity, but during the whole of one's zooming and scrolling to new locations the URL remains the same. Google Map's solution is to provide a "Link to this page" hyperlink (in my opinion needs to be brought to the visual forefront a little better as I have problems getting people to recognize the link when they have sent me a link to maps.google.com rather than their intended page).

Current examples of a poor grasp of state is found on the DUX 2005 conference site. Every page has the same URL, from the home page, to submission page, to about page. You can not bookmark the information that is important to yourself, nor can you send a link to the page your friend is having problems locating. The site is stateless in all of its failing glory. The designer is most likely not clueless, just thoughtless. They have left out the person using the site (not users, as I am sure their friends whom looked at the design thought it was cool and brilliant). We have to design with people using and resusing our site's information in mind. This requires state.

When is State Helpful?

If you have important information that the people using your site may want to directly link to, state is important as these people will need a URL. If you have large datasets that change over time and you have people using the data for research and reports, the data must have state (in this case it is the state of the data at some point in time). Data that change that does not have state will only be use for people that enjoy being selected as a fool. Results over time will change and all good academic research or professional researchers note the state of the data with time and date. All recommendations made on the data are only wholly relevant to that state of the data.

Nearly all blogging tools have "permalinks", or links that link directly to an unchanging URL for distinct articles or postings, built into the default settings. These permalinks are the state function, as the main page of a blog is fluid and ever changing. The individual posts are the usual granular elements that have value to those linking to them (some sites provide links down to the paragraph level, which is even more helpful for holding a conversation with one's readers).

State is important for distinct chunks of information found on a site. Actions do not seem state-worthy for things like uploading files, "loading screens", select your location screens (the pages prior and following should have state relative to the locations being shown on those pages), etc.

The back button should be a guide to state. If the back button takes the user to the same page they left, that page should be addressable. If the back button does not provide the same information, it most likely should present the same information if the person using the site is clicking on "next" or "previous". When filling out an application one should be able to save the state of the application progress and get a means to come back to that state of progress, as people are often extremely aggravated when filling out longs forms and have to get information that is not in reach, only to find the application times out while they are gone and they have to start at step one after being many steps into the process.

State requires a lot of thought and consideration. If we are going to build the web for amateurization or personal information architectures that ease how people build and structure their use of the web, we must provide state.

Removing the Stench from Mobile Information

Standing in Amsterdam in front of the Dam, I was taking in the remnants of a memorial to Theodore van Gogh (including poetry to Theo). While absorbing what was in front of me, I had a couple people ask me what the flowers and sayings were about. I roughly explained the street murder of Theo van Gogh.

While I was at the Design Engaged conference listening to presentations about mobile information and location-based information I thought a lot about the moment at the Dam. I thought about adding information to the Dam in an electronic means. If one were standing at the Dam you could get a history of the Dam placed by the City of Amsterdam or a historical society. You could get a timeline of memorials and major events at the Dam. You could also get every human annotation.

Would we want every annotation? That question kept running reoccurring and still does. How would one dig through all the digital markings? The scent of information could become the "stench of information" very quickly. Would all messages even be friendly, would they contain viruses? Locations would need their own Google search to find the relevant pieces of information. This would all be done on a mobile phone, those lovely creatures with their still developing processors.

As we move to a world where we can access information by location and in some cases access the information by short range radio signals or touching our devices there needs to be an easy to accept these messages. The messaging needs some predictive understanding on our mobiles or some preparsing of content and messaging done remotely (more on remote access farther down).

If was are going to have some patterning tools built in our mobiles what information would they need to base predictions? It seems the pieces that could make it work are based on trust, value, context, where, time, action, and message pattern. Some of this predictive nature will need some processing power on the mobile or a connection to a service that can provide the muscle to predict based on the following metadata assets of the message.

Trust is based on who left the message and whether you know this person or not. If the person is known do you trust them? This could need an ensured name identification, which could be mobile number, their tagging name crossed with some sort of key that proves the identity, or some combination of known and secure metadata items. It would also be good to have a means to identify the contributor as the (or an) official maintainer of the location (a museum curator annotating galleries in a large museum is one instance). Some trusted social tool could do some predicting of the person's worthiness to us also. The social tools would have to be better than most of today's variants of social networking tools as they do not have the capability for us to have a close friend, but not really like or trust their circle(s) of friends. It would be a good first pass to go through our own list of trusted people and accept a message left by any one of these people. Based on our liking or disliking of the message a rating would be associated with this person to be used over time.

Value is a measure of the worthiness of the information, normally based on the source of the message. Should the person who left the message have a high ranking of content value it could be predicted that the message before us is of high value. If these are message that have been reviews of restaurants and we have liked RacerX previous reviews we found in five other cities and they just gave the restaurant we are in front of a solid review that meets our interests. Does RacerX have all the same interests?

Context is a difficult predictive pattern as there are many contextual elements such as mood, weather, what the information relates to (restaurant reviews, movie reviews, tour recommendations, etc.). Can we set our mood and the weather when predicting our interest in a message. Is our mood always the same in certain locations?

Where we are is more important than location. Yes, do we know where we are? Are we lost? Are we comfortable where we are? These are important questions that may help be a predictor that are somewhat based on our location. Or location is the physical space we occupy, but how we feel about that spot or what is around us at that spot may trigger our desire to not accept a location-based message. Some of us feel very comfortable and grounded in any Chinatown anywhere around the globe and we seek them out in any new city. Knowing that we are in or bordering on a red-light district may trigger a predictive nature that would turn off all location-based messages. Again these are all personal to us and our preferences. Do our preferences stay constant over time?

Time has two variables on two planes. The first plane is our own time variables while the other relates to the time of the messages. One variable is the current moment and the other is historical time series. The current moment may be important to us if it is early morning and we enjoy exploring in the early morning and want to receive information that will augment our explorative nature. Current messages may be more important than historical messages to us. The other variable of historical time and how we treat the past. Some of us want all of our information to be of equal value, while others will want the most current decisions to have a stronger weight so that new events can keep information flowing that is most attune to our current interests and desires. We may have received a virus from one of our recent messages and want to change our patterns of acceptance to reflect our new cautionary nature. We may want to limit how far back we want to read messages.

Action is a very important variable to follow when the possibility of malicious code can damage our mobile or the information we have stored in the mobile or associated with that mobile. Is the item we are about to receive trigger some action on our device or is is a static docile message. Do we want to load active messages into a sandbox on our mobile so the could not infect anything else? Or, do we want to accept the active messages if they meet certain other criteria.

Lastly, message pattern involved the actual content of the message and would predict if we would want to read the information if it is identical or similar to other messages, think attention.xml. If the Dam has 350 messages similar to "I am standing at the Dam" I think we may want to limit that to ones that meet some other criteria or to just one, if we had the option. Do we have predictors that are based on the language patterns in messages? Does our circle of trusted message writers always have the same spellings for certain wordz?

All of these variables could lead to a tight predictive pattern that eases the information that we access. The big question is how is all of this built into a predictive system that works for us the moment we get our mobile device and start using the predictive services? Do we have a questionnaire we fill out that creates our initial settings? Will new phones have ranking buttons for messages and calls (nice to rank calls we received so that our mobile would put certain calls directly into voice mail) so it is an easier interface to set our preferences and patterns.

Getting back to remote access to location-based information seems, for me, to provide some excellent benefits. There are two benefits I see related to setting our predictive patterns. The first is remote access to information could be done through a more interactive device than our mobile. Reading and ranking information from a desktop on a network or a laptop on WiFi could allow us to get through more information more quickly. The second benefit is helping us plan and learn from the location-based information prior to our going to that location so we could absorb the surroundings, like a museum or important architecture, with minimal local interaction with the information. Just think if we could have had our predictive service parse through 350 messages that are located at the Dam and we previews the messages remotely and flagged four that could have interest to us while we are standing at the Dam. That could be the sweet smell of information.

Web 2.0: Source, Container, Presentation

At Web 2.0 Jeff Bezos, of Amazon stated, "Web 2.0 is different. It's about AWS (Amazon Web Services). It's not on the web site for users to see. It's about making the internet useful for computers.". This is very appropriate today as it breaks the information model into at least three pieces: source, container, and presentation. Web 1.0 often had these three elements in one place, which really made it difficult to reuse the information, but even use it at times.

The source is the raw information or content from the creator or main distributor. The container is the means of transporting the information or content. The container can be XML, CSV, text, XHTML, etc. The presentation is what is used to make the information or content human consumable. The presentation can be HTML with CSS, Flash, PDF, feed reader, mobile application, desktop application, etc.

The importance of the three components is they most valuable when they stand alone. Many problems and frustrations for people trying to get information and reuse it off the web has been there has not been a separation of the components. Take most Flash files, which tie the container and the presentation in one object that is proprietary and can be extremely difficult to extract the information for reuse. The same also applies to PDF files as they too are less than optimal for sharing information for anything other than reading, if the PDF can be read on the device. As mobile use of the internet increases the separation is much more valuable. The separation has always been the smart thing to do.

Today Google launched a beta of their Google SMS for mobile devices. The service takes advantage of the Google web services (source) and allows mobile users to send a text message with a query (asking "pizza" and providing the zip code) and Google responds with a text message with information (local pizzerias with their address and phone numbers). The other day Tantek demonstrated Semantic XHTML as an API, which provides openly accessible information that is aggregated and reused with a new presentation layer, Flash.

More will follow on this topic at some point in the not too distant future, once I get sleep.

Quick Links are Now Pulled from del.icio.us

We did a little house keeping today. We have been using our del.icio.us link tool as a replacement for the Quick Links. Today the Quick Links are pulled from del.icio.us. We found that as we came across helpful links through out the day del.icio.us made it easy to add links and information. The del.icio.us tool also has made it easy to find similar information.

We extracted the vanderwal del.icio.us links from del.icio.us using their del.icio.us API and formated the information in a very similar manner to what was displayed in the Quick Links previously in the side bar. The del.icio.us Web Service is wonderful to work with and made this task possible.

We are hoping that over the weekend we will put up a page of all the previous Quick Links. We have it in our plans to add all the entries on this site into del.icio.us directly using this site's categories. The del.icio.us flat category structure is similar what we built here in 2001. The categories between this site and what we keep at del.icio.us do not synch so we do not have the categories from del.icio.us stored here. There is always something more to do.

We are also considering moving the quick links out of the side bar and place them in the main content area in a grouping. This will take some thinking. If you have ideas on this front please contact us.

Feed On This

The "My" portal hype died for all but a few central "MyX" portals, like my.yahoo. Two to three years ago "My" was hot and everybody and their brother spent a ton of money building a personal portal to their site. Many newspapers had their own news portals, such as the my.washingtonpost.com and others. Building this personalization was expensive and there were very few takers. Companies fell down this same rabbit hole offering a personalized view to their sites and so some degree this made sense and to a for a few companies this works well for their paying customers. Many large organizations have moved in this direction with their corporate intranets, which does work rather well.

Where Do Personalization Portals Work Well

The places where personalization works points where information aggregation makes sense. The my.yahoo's work because it is the one place for a person to do their one-stop information aggregation. People that use personalized portals often have one for work and one for Personal life. People using personalized portals are used because they provide one place to look for information they need.

The corporate Intranet one place having one centralized portal works well. These interfaces to a centralized resource that has information each of the people wants according to their needs and desires can be found to be very helpful. Having more than one portal often leads to quick failure as their is no centralized point that is easy to work from to get to what is desired. The user uses these tools as part of their Personal InfoCloud, which has information aggregated as they need it and it is categorized and labeled in a manner that is easiest for them to understand (some organizations use portals as a means of enculturation the users to the common vocabulary that is desired for use in the organization - this top-down approach can work over time, but also leads to users not finding what they need). People in organizations often want information about the organization's changes, employee information, calendars, discussion areas, etc. to be easily found.

Think of personalized portals as very large umbrellas. If you can think of logical umbrellas above your organization then you probably are in the wrong place to build a personalized portal and your time and effort will be far better spent providing information in a format that can be easily used in a portal or information aggregator. Sites like the Washington Post's personalized portal did not last because of the cost's to keep the software running and the relatively small group of users that wanted or used that service. Was the Post wrong to move in this direction? No, not at the time, but now that there is an abundance of lesson's learned in this area it would be extremely foolish to move in this direction.

You ask about Amazon? Amazon does an incredible job at providing personalization, but like your local stores that is part of their customer service. In San Francisco I used to frequent a video store near my house on Arguello. I loved that neighborhood video store because the owner knew me and my preferences and off the top of his head he remembered what I had rented and what would be a great suggestion for me. The store was still set up for me to use just like it was for those that were not regulars, but he provided a wonderful service for me, which kept me from going to the large chains that recorded everything about me, but offered no service that helped me enjoy their offerings. Amazon does a similar thing and it does it behind the scenes as part of what it does.

How does Amazon differ from a personalized portal? Aggregation of the information. A personalized portal aggregates what you want and that is its main purpose. Amazon allows its information to be aggregated using its API. Amazon's goal is to help you buy from them. A personalized portal has as its goal to provide one-stop information access. Yes, my.yahoo does have advertising, but its goal is to aggregate information in an interface helps the users find out the information they want easily.

Should government agencies provide personalized portals? It makes the most sense to provide this at the government-wide level. Similar to First.gov a portal that allows tracking of government info would be very helpful. Why not the agency level? Cost and effort! If you believe in government running efficiently it makes sense to centralize a service such as a personalized portal. The U.S. Federal Government has very strong restriction on privacy, which greatly limits the login for a personalized service. The U.S. Government's e-gov initiatives could be other places to provide these services as their is information aggregation at these points also. The downside is having many login names and password to remember to get to the various aggregation points, which is one of the large downfalls of the MyX players of the past few years.

What Should We Provide

The best solution for many is to provide information that can be aggregated. The centralized personalized portals have been moving toward allowing the inclusion of any syndicated information feed. Yahoo has been moving in this direction for some time and in its new beta version of my.yahoo that was released in the past week it allows the users to select the feeds they would like in their portal, even from non-Yahoo resources. In the new my.yahoo any information that has a feed can be pulled into that information aggregator. Many of us have been doing this for some time with RSS Feeds and it has greatly changed the way we consume information, but making information consumption fore efficient.

There are at least three layers in this syndication model. The first is the information syndication layer, where information (or its abstraction and related metadata) are put into a feed. These feeds can then be aggregated with other feeds (similar to what del.icio.us provides (del.icio.us also provides a social software and sharing tool that can be helpful to share out personal tagged information and aggregations based on this bottom-up categorization (folksonomy). The next layer is the information aggregator or personalized portals, which is where people consume the information and choose whether they want to follow the links in the syndication to get more information.

There is little need to provide another personalized portal, but there is great need for information syndication. Just as people have learned with internet search, the information has to be structured properly. The model of information consumption relies on the information being found. Today information is often found through search and information aggregators and these trends seem to be the foundation of information use of tomorrow.

Rael on Tech

Tech Review interviews Rael about rising tech trends and discusses alpha geeks. This interview touches on RSS, mobile devices, social networks, and much more.

Matt on Social Networks

Matt writes up this thoughts on the state of social networks. I agree with much of his frustration. I keep thinking many of these tools will provide some good value. The two that meet what I expect are Upcoming and LinkedIn. I like these are they offer small targeted offerings. Upcoming helps find and track events, while LinkedIn is a work related networking tool.

It seems a simple cross between LinkedIn and XFN or any metadata resource that can track relationships, trust, and taste along with tracking other items of interest would be greatly helpful. Matt does get the metadata problem included in his write-up, which is metadata is dirty and at best, biased (which can be good if you agree with the bias).

CSS in RSS

Something to come back to, CSS in RSS. It works in NetNewsWire.

Interdependance of structure, information, and presentation

Peter J. Bogaards explains The Document Triangle: The interdependence of the structure, information and presentation dimensions. This troika is very important clear information consumption, but also information reuse. Structure is extremely important to transmitting information, but also important to information reuse. Information lacking structure nearly as reusable as a newspaper article printed on paper.

One great location to explore the ease of information reuse and the affect the presentation layer has should look no farther than, CSS Zen Garden, where nearly all the content is identical in the various layouts and designs. The structure of the content provides a solid framework to rework the presentation layer. The presentation layer can add to or detract from the clarity of the message as well as the attraction a user may have to the message.

Information structure important for information reuse

John Udell's discussion of Apple's Knowledge Navigator is a wonderful overview of a Personal Information Cloud. If the tools was more mobile or was shown synching with a similar mobile device to have the "knowledge" with the user at all time it is would be a perfect representation.

Information in a Personal Information Cloud is not only what the user wants to have stored for retrieval when it is needed (role-based information and contextual) but portable and always accessible. Having tools that allow the user to capture, categorize, and have attracted to the user so it is always with them is only one part of the equation. The other component is having information that is capable of being captured and reused. Standards structures for information, like (X)HTML and XML are the beginnings of reusable information. These structures must be open to ensure ease of access and reuse in proper context. Information stored in graphics, proprietary software, and proprietary file formats greatly hinders the initial usefulness of the information as it can be in accessible, but it even more greatly hinders the information's reuse.

These principle are not only part of the Personal Information Cloud along with the Model of Attraction, but also contextual design, information architecture, information design, and application development.

RSS on PDAs and information reuse

Three times the past week I have run across folks mentioning Hand/RSS for Palm. This seems to fill the hole that AvantGo does not completely fill. Many of the information resources I find to be helpful/insightful have RSS feeds, but do not have a "mobile" version (more importantly the content is not made with standard (X)HTML validating markup with a malleable page layout that will work for desktop/laptop web browsers and smaller mobile screens).

I currently pull to scan then read content from 125 RSS feeds. Having these some of these feeds pulled and stored in my PDA would be a great help.

Another idea I have been playing with is to pull and convert RSS feeds for mobile browser access and use. This can be readily done with PHP. It seems that MobileRSS already does something like this.

Content, make that information in general, stored and presented in a format that is only usable in one device type or application is very short sighted. Information should be reusable to be more useful. Users copy and paste information into documents, todo lists, calendars, PDAs, e-mail, weblogs, text searchable data stores (databases, XML respositories, etc.), etc. Digital information from the early creation was about reusing the information. Putting text only in a graphic is foolish (AIGA websites need to learn this lesson) as is locking the information in a proprietary application or proprietary format.

The whole of the Personal Information Cloud, the rough cloud of information that the user has chosen to follow them so that it is available when they need that information is only usable if information is in an open format.

Bray's escaping for RSS

Tim Bray discusses escaping characters in RSS feeds

Steve Champeon on the Future of Web Design

Steve Champeon on Progressive Enhancement and the Future of Web Design. This is almost like sitting with Steve and getting the background and how that reflects for future of markup and Web design directly from Steve.

OmniOutliner updated with Visio import and export

OmniGraffle 3.0 is out today as is the Pro version. The Pro version has import and export of files with Visio 2002 (using the Visio XML format) and includes mouseless editing (these two features could make the Pro version worth it for me). The interface has received an complete redesign and is much better incorporated into OS X, which version 2 ran very well under. The regular version will output files to PDF, PNG, JPEG, and HTML (among others), if the HTML is as clean as the OmniOutliner this will be a treat. I have been looking forward to this upgrade of one of my favorite tools.

Powells Books Booty

Okay, here is the list of booty from Powells Books... Metarepresentations: a multidisciplinary prespective edited by Dan Sperber, a description of this Cog Sci overview book help understand it better. Kunstler's The City in Mind. Feynman's Six Easy Pieces, which I started this morning on the train and really enjoy. William Gibson's Pattern Recognition that I started reading on the plane and has really pulled me in. A string of tech books, MySQL Cookbook, Perl and XML, and Java and XML, and based on Peter's recommendation Managing Enterprise Content: A Unified Content Strategy. This and yesterday's mentioned Hofstadler book should about cover it. I really wish there were a Powells Books where I lived, but my wallet does not wish the same. It is great to be able to see the books and evaluate how helpful the book will actually be to you before buying.

I also added Information Architecture: An Emerging 21st Century Profession by Earl Morrogh while at the IA Summit. It seems to be a very good overview on first pass and it comes very highly recommended. I met Earl at the Summit and he is purely delightful and very much a part of the IA community.

Konfabulator is fabu baby

I finally downloaded Konfabulator and I am having fun. I am impressed with the widgets (small single purpose applications that elegantly sit on the desktop of a Mac). Go check it out, you may find that small application you are looking for, or you may create it with XML and JavaScript (what all the widgets are made with).

XML Schema for Apple Keynote

The XML schema for Apple's Keynote explained in Technotes. This may come in handy in the not so distant future. [thanks to Daniel Steinberg for the heads up].

Apple Word Replacement Rumor and Information Structure Dreams

Rumor has it Apple is working on MS Word replacement. This would be a great thing if it would read native Word files seemlessly, but even better would be turning out valid HTML/XHTML. MS Word has always made a huge mess of our information with its conversion to something it "calls" HTML, it is not even passable HTML. One could not get a job using what Microsoft outputs as HTML as a work sample, heck, it would not even pass the laugh test and it may get somebody fired.

One of the downsides of MS Office products is that they are created for styling of information not marking up information with structure, to which style can hang. MS Word allows people (if the turn on or keep the options turned on) to create information sculptures with structure and formatting of the information. What Word outputs to non-Word formats is an information blob that has lost nearly all of its structure and functionality in any other format. It does not really have the format the Word document to begin with. What Web developers do is put the structure back into the information blob to recreate an information sculpture again.

You ask why is structure important? Structure provides the insight to know what is a header and sub-header. Structure provides the ability to discern bulleted lists and outlines. Structure makes it script-kiddie easy to create a table of contents. Structure makes micro-content accessible and easier to find with search. Structure provides better context. Structure provides the ability to know what is a quote from an external document and point to it easily. Structure provides ease of information portability and mobile access easier. These just name a few uses of structure.

Does MS Word have this structure capability? Yes, do people use it? No really. If people use it does MS Word keep the structure? Rarely, as it usually turns the structure into style. This is much like a somebody who spent months in the gym to build a well defined physique only to have the muscles removed to stuff their own shirt with tissue paper to give it the look of being in shape. Does the person with the tissue paper muscles have the ability to perform the same as the person who is really in shape? Not even close.

Structure is important not only for the attributes listed above, but also for those people that have disabilities and depend on the information being structured to get the same understanding as a person with out disabilities. You say MS Word is an accessible application, you are mostly correct. Does it create accessible information documents? Barely at best. The best format for information structure lay in HTML/XHTML/XML not in styles.

One current place that structure is greatly valuable is Internet search. Google is the top search engine on the Internet. Google uses the text in hyperlinks, the information in title tags, and information in the heading tags to improve the findability of a Web page. What are these tagged elements? Structure.

One of the nice things about a valid HTML/XHTML Web document is I can see it aqnd use it on my cell phone or other mobile devices. You can navigate without buttons and read the page in chunks. Some systems preparse the pages and offer the ability to jump between headings to more quickly get to the information desired.

These are just a few reasons I am intrigued with the Apple rumor. There is hope for well structured documents that can output information in a structured form that can validate to the W3C standards, which browsers now use to properly render the information on the page. I have very little hope in the stories that MS is working toward an XML storage capability for Office documents, because we have heard this same story with the last few Office releases and all were functional lies.

Facets made usable with XFML

Introduction to XFML by Peter Van Dijck is a great first step to understanding the eXtreme Football Markup Language eXchangeable Faceted Metadata Language. This article lays out facets, topics, metadata, taxonomy, and provides a framework based on these understandings to build a usable structure to describe objects.

Parse RSS even if it is wrong

Parse RSS at All Cost by Mark Pilgrim what is required to parse RSS properly. More importantly Mark points out that as more RSS feeds are created the feeds are being poorly created. Then Mark instructs how to build a parser that will be a little more forgiving of poor markup.

RSS feeds are very Clue Train friendly it seems

Not long after I posted my RSS disconnecting the creator and the user comments it started sinking in that it really does not matter. We it does to some part, but from a user's perspective the RSS allows a quicker more efficient method of scanning for information they have an interest in and easily see from one interface when new content has been written. I use other's blogs and digests to find information to post for my own reflection and to use as jumping boards to new ideas.

Yes, the interaction between creator and user is important, but it is not as important as getting informtion out. I began thinking that the whining about the lack of interaction on my part was rather selfish and very contrary to the focus I have for most information, which is having the abiltiy to access, digest, add to, or reformulate the information into another medium or presentation that will offer possibly better understanding.

I was self-taught in the values of the Clue Train so when I heard about it for the first time I was supprised so some large degree that the manifesto had resonance and turned on a light for many people, for myself and some others, I guess we drank the cool-aide early, as we thought this was the way things were or should be from the beginning of electronic information and a truely open community where information flows freely. Yes, the RSS/RDF/XML feed is a freer flow of information and puts the choice of the information consuption in the user's hands.

RSS and interconnections

Since I added the vanderwal.net RSS feed I have been picking up other RSS and RDF feeds. I have been using Ranchero's NetNewsWire Lite to pull many feeds of sites I read on a regular basis. I have become a convert to RSS/RDF extracts. They are a time saver for seeing only updated sites. I have read feeds of many of the news sites from MacReporter for quite sometime, but having personal content and blogs pulled in is quite a timesaver and allows me to get through more information.

I do see a downside of the XML feeds, in the disconnection of the creator from the users. The Web has given us the ability to have digital ghosts that we know come to our sites and possibly read content. This is much like Plato's cave shadow people, in that we do not see the actual people that come to the sites, but we surmise what these visitors are like and what they come to read. Occasionally we receive comments on the site, e-mails from visitors, or best meet folks in person that read/experience your work. It is very much a disconnected work that is built from guesses, for those that try and care (some just build for themselves resources to be used remotely and all others are welcome "free riders", like here). The XML feeds seem to take away another level of the "interaction" between the creator and the users. This relationship is important in communication as the feedback helps shape the message as well as offer paths for both parties to learn and grow.

The XML feeds offer the consumers of the information easier and more efficient means of getting, filtering, and digesting information, but the return path to the creator is diminished. The feeds are a consumer oriented communication channel and not so much an interactive communiction channel. The down side is a lack of true interactive communication, which becomes more of a consuming produced products, much like frozen dinners that get popped in the microwave. The interaction provides the creator with an understanding of how the user consumes the information and what the consumer of the information is finding usable and how the consumer is being drawn to the information. When one cooks their own meals or is being cooked for the meal can be spiced and seasoned appropriately for consumption. The presentation of the food can be modified to enhance pleasure. The live cooking process allows for feedback and modification. Much like the interaction of information in a communication scenario the creator and the consumer have a relationship, as the creator finds the structure and the preferred means of consuming the information the presentation and structure of the information can be altered appropriately.

In a sense the XML feed could be seen as one type of information structure of presentation. There are other options available that can be used to bring back the interaction between the creator and consumer. Relationships and connections are built over this expansive medium of the Web through information and experience. These connections should be respected and provided a place to survive.

Mark dives into XFML

Mark Pilgim dives into XFML and provides an excellent set of references to learn more (as always).

W3C RDF Primer

The W3C RDF Primer is something to come back to soon. The Resource Description Framework is a solid foundation to sharing information and is getting used more. It is a grown-up's version of RSS (the weblogger's resource sharing XML tool). This information relies on well structured information and helps keep the information structured for reuse.

Get your RSS feed

Yes, I finally got up to speed with the rest of the world and added an RSS feed and have added a new page that will track available vanderwal.net XML documents and RSS feeds. I may make a couple category specific RSS feeds as there is interest. Use the (now working again) comments or the contact to let me know what you would like.

I have only put out the first RSS feed in 0.91 at the moment. I may upgrade it in the near future as I now have it relatively easy to build from my end. I have been getting a decent amount of pestering and bother from folks asking for the feed. You see I still build my own CMS for the site and it takes time and priority to get around to some of these things.

Why not move to Movable Type or Drupal (the only two I would currently consider)? I enjoy building my own system, but it does require that I build my own APIs or even my own applications to mirror functionality. I like building CMS and this one is one of six that I have designed and/or fully built since 1997. It is like a hobby for me as well as a job.

RDF is more than three letters

You need to buy a vowel to make a word with the letters RDF, but RDF can be used to share informaiton and communicate. Kendall Grant Clark explains the uses for the Resource Description Framework.

RSS in the near future

The RSS feed is in the works here and should be ready by the anniversary of the tool, which is October 31st. There have been many upgrades to the application that runs this site in the past year, but one that I really wanted was RSS. I have it running on my laptop and it needs only one tweak and then to get dropped into my main posting page. Did I mention I am loving having my a my development tools and full webserver with all the application elements running on my TiBook laptop again?

Microcontent browser

Anil's Dashes Magazine discusses microcontent browsers, which is a great idea that is on the cusp of being here. Anil outlines what is needed and where it will all end up, hopefully.

Topic Maps Explained

Topic Maps explained over at O'Reilly Net.

Business Maps

Over at O'Reilly Net Marc De Graauw puts forth Business Maps: Topic Maps Go B2B, which seems to be an IAs dream (that would be an enterprise IA or a Macro IA, which are the ones that perform extra enterprise IA). The article also points to an overview of Topic Maps. Good stuff.

Standard Data Vocabularies Unquestionably Harmful

Must come back to this when my mind is fresh, Standard Data Vocabularies Unquestionably Harmful over at O'Reilly Net. This seems right up the alley for an IA.

Emerging Tech Conference coverage

There is a lot of amazing things that have happened at the Emerging Tech Conference 2002. Great minds discussing great things, what could be better.

Blosxom offers RSS aggregation with perl

Blosxom is a Perl RSS blog aggregator that works well on Mac OS X.

Expat help for Mac

An overview of expat will help tie the loose ends together. For those Mac heads reading you may also want to take in Life with CPAN by Jeremy Mates which puts together the missing pieces for Mac.

Perl and XML from the command line

While we are on the Perl and XML kick, O'Reilly offers Perl and XML on the Command Line.

Perl XML to take advantage of Amazon

Amazox is just what you need to take advantage of Amazon Associates' SML. [hat tip Michael]

XML for org charts and so much more

Thanks to Anil, I came across An open toolkit to facilitate knowledge extraction and document analysis, which contains a solid approach to org charts. I know quite a few folks that sing the "Org Chart Blues" on a daily basis. This is a solid step in the right direction. This document is the work of Gareth Hughes

One thing that I have had the benefit of viewing and discussing while at SXSWi is Flash. Folks from Macromedia have shown their soon to be released version of Flash. Flash MX (as it is named) has some very good new features, in its providing common Web development objects to help developers create scroll bars and the like very easily. The application seems to provide object or extensions to Flash that streamline the process to building something usable and consistent in Flash.

The best new feature of Flash is content can now be made accessible for those with sight disabilities. This is greatly helpful as Flash is largely a visual information development and presentation tool. The information is now usable by site readers that read information aloud stored in Flash. This has been a large hinderance for many folks who would like to adopt Flash into their development tool belt, but had restrictions that limited the use of Flash because it locked out a segment of the users who had visual disabilities.

There is one large element in Flash that is completely disappointing still. The information is not accessible for reuse. All Flash can provide is visual information presentation, which restricts a user's ability to copy and paste or to have the information machine readable. The information is locked in an unusable format for these purposes. What does this affect? If a hotel provides their phone number and address in their Flash presentation the user can not copy and paste the information out into their PIM (Outlook, Access, Palm desktop, etc.), to an e-mail, or text message that the user could read from their PDA or cell phone (given messaging capabilities). The user would also be restricted from grabbing the information to put together a matrix from which to make decisions or to supply to others to make their comparisons. The locking of information in Flash requires the user to retype the information provided, which introduces the ability for errors in the information that was carefully crafted.

Not only do human users have the inability to re-purpose the information, which is a great benefit to those providing the information, but machines are precluded from accessing the information. If the same hotel wants to be included in their city's chamber of commerce listing on the CCoC Web site the hotel information can not be easily extracted by the CCoC as it could be from HTML (using an id tag) or XML. The information is locked again in an unusable and un-reusable format. The creators of the content lose, and could possibly lose big by not having information that is easily reused. This becomes increasingly important with the growing use of Web Services that rely on machine readable or machine accessible information.

Why the hotel scenario? Macromedia used the hotel demonstration to highlight some of their great new features. As I watched the presentation I kept wondering if the information was still unusable for purposes other than reading or having the information read to the user. It was later confirmed the information was still un-reusable, but Macromedia is also aware of this strong down side to the information presentation and is working hard at fixing the issues.

The Visual Display of Quantitative XML on O'Reilly Net really rocks for me. I am really impressed with the presentation, but not nearly as impressed as I was with the ease of downloading and running the SVG plug-in in IE 6 on Windows and IE 5.1 on Mac OS X. Overal this is a great article as it not only walks through the how-to portion, but also offers insights into things that will make similar development go more easily.

Representational State Transfer (REST) and the Real World provides the ability to add security to XML-based Web Services, among other beneficial elements.

The school of hard knocks teaches us to learn where the pitfalls may lie in our plans as we are in the discovery and planning stages so we do not add to our "lessons learned". Web Services Pitfalls from O'Reilly Net and XML.com provides this wonderful bruise saving insight.

Okay, O'Reilly Net has been offering a lot of good resources of late and I am a little behind in catching up.

Those of us that need a light weight database for a small project and are using Microsoft Windows often turn to Access to perform the task. Steven Roman offers his tips in how to set up an Access database. The best tip is right up front:

Don't touch your PC until you have taken the time to rough out a design of your database.

The tips keep coming and many of them apply to any other database development. Once I learned to think of information and metadata in a relational database format (which also helps with setting up XML documents) application development became easier. Understanding a little database design will also help ask the right questions when setting up an application, database, and/or project.

This article also helps define the limitations of Access databases. Each database will have its own limitations or peculiarities. Knowing these differences will help think about the application, information, and how they can and will be used is helpful.

The U.S. Government urges to focus its tech developers to focus on standards for XML. This and more is stated discussed in this O'Reilly Net article.

Web Services Interoperability from James†Snell on O'Reilly Net's XML.Com. This wonderful piece demonstrates, with code, the interaction that is currently possible with standard development environments. This is familiar territory for me from a recent past life, although it was not using SOAP, like these examples do.

IBM explains RDF in this wonderful introduction. [hat tip Nick at Digital Web New]

Content management is back at the forefront of every aspect of my digital life again. Content management revolves around keeping information current, accurate, and reusable (there are many more elements, but these cut to the core of many issues). Maintaining Websites and providing information resources on the broader Internet have revolved around static Web pages or information stored in MS Word, PDF files, etc. Content management has been a painful task of keeping this information current and accurate across all these various input and output platforms. This brings us to content management systems (CMS).

As I pointed to earlier, there are good resources for getting and understanding CMS and how our roles change when we implement a CMS. Important to understanding is the separation of content (data and information), from the presentation (layout and style), and from the application (PDF, Web page, MS Word document, etc.). This requires an input mechanism, usually a form that captures the information and places it in is data/information store, which may be a database, XML document, or a combination of these. This also provides for a workflow process that involved proofing and editing the information along with versioning the information.

Key to the CMS is separation of content, which means there needs to be a way to be a method of keeping links aside from the input flow. Mark Baker provides a great article, What Does Your Content Management System Call This Guy about how to handle links. Links are an element that separates the CMS-lite tools (Blogger, Movable Type, etc.) from more robust CMS (other elements of difference are more expansive workflow, metadata capturing, and content type handling (images, PDF, etc. and their related metadata needs)). Links in many older systems, often used for newspaper and magazine publications (New York Times and San Francisco Chronicle) placed their links outside of the body of the article. The external linking provided an easy method of providing link management that helps ensure there are no broken links (if an external site changes the location (URL) it there really should only be one place that we have to modify that link, searching every page looking for links to replace). The method in the Baker article outlines how many current systems provide this same service, which is similar to Wiki Wiki's approach. The Baker outlined method also will benefit greatly from all of the Information Architecture work you have done to capture classifications of information and metadata types (IA is a needed and required part of nearly every development process).

What this gets us is content that we can easily output to a Web site in HTML/XHTML in a template that meets all accessibility requirements, ensures quality assurance has been performed, and provides a consistent presentation of information. The same information can be output in a more simple presentation template for handheld devices (AvantGo for example) or WML for WAP. The same information can be provided in an XML document, such as RSS, which provides others access to information more easily. The same information can be output to a template that is stored in PDF that is then sent to a printer to output in a newsletter or the PDF distributed for the users to print out on their own. The technologies for information presentation are ever changing and CMS allows us to easily keep up with these changes and output the information in the "latest and greatest", while still being able to provide information to those using older technologies.

Need to grab information out of an MS Excel spreadsheet with out all the display information? You may want to pull Excel information out to XML as this would provide very clean information. From this point if you wanted to use this information in an HTML page or another output format it would be a rather straightforward parse. This could be a good tool to try and work with.

After procrastinating for long enough and reading Nick's review in Digital Web I upgraded to Homesite 5. This is what I used to update and validate sections of this site to XHTML. I have been using Homesite for work projects for a few years, but rarely used it for this site or other personal projects (just a quirk) as I usually do my work handcoding with TextPad. Some of the layout of the tools has changed slightly from Homesite 4.5.2 to 5, but it was not a major difference. I really liked the XHTML elements and collapsing the code, which makes finding non-closed tags an easy task.

I have been able to read through all of this month's Digital Web and can say it is a solid issue from end to end. I really enjoyed the interview with Hether Hesketh. I am also a fan of the two business pieces, Managing the client: A fairy tale and Building the Business Game Plan. Both of these business articles I have pointed others to already as they are great insights from experience.

Troy Janish's XML tutorial for A List Apart is a great resource. The elements for implementing XML are rather solid now. Knowing how to make use of XML is going to be a needed skill set for many. The debate over database storage of content objects over XML storage of the same will continue for quite some time. I have my leaning for database storage, given their data storage and manipulation functionality built in, but outputing XML documents to share the information seems to be a better option than providing authorized access to the database.

3D XML viewer/navigator, built in Flash. This is brough to us by David Lu. [hat tip Jason]

Moving to XHTML and general updates

There are some changes around here. The links page has been updated with some new links, updated links, and a few removed (ones that I was not visiting for various reasons or had gone dead).

The links and about pages are both converted to XHTML and are validating, for the most part, to XHTML Transitional. The next step will be to get this section, Off the Top, to validate. This will be a little more effort as it will require making some edits to the templates and internal code validation. Not a monsterous task, but a task none-the-less. A large part of the conversion in this section is creating compliant output from non-standard input. Much of this section does not use starting paragraph tags (<p>), which will take some work to ammend.

This means that this site is finally moving toward being standards compliant. This means that it will be easier to display information across browsers (standards compliant browsers, which most are becoming), ease of maintenance, and information reuse.

A Nick posts the outline and links for he and Ross Olson's Web Standards discussion, which was delivered in Portland, Oregon at the Portland Multimedia | Internet Developer's Group.

Triggered by Kendall Grant Clarks review of Scrollkeper on O'Reilly Net I finally got back to the Open Source Metadata Framework. I was following some of the early development on this as they were looking at embracing Dublin Core. The result is a nice malleable format that can be wrapped around many ideas. It is an essential component of Content Management metadata. Now matter how you twist CM these elements should be there in one form or another.

The XSLT Standard Library makes XSLT templates of commonly used functions. [hat tip Bill Humpheries]

Are you using a Mozilla based browser, such as Netscape 6x? You may want to grab the XUL useragent toolbar, which allows you to overwrite your general.useragent (the component in the Web browser that states its browser type to the Web sites). Why is this important? If you are using Netscape 6x or Mozilla and the folks maintaining the site have not added the proper browser sniffing to include these browsers you may not find the site usable. As NS6x and Mozilla are Web standards compliant, as is IE 5.5 and up, you should be able to set your useragent to reflect IE 5.5 or IE 6 and find the site perfectly usable.