Off the Top: Scripting Entries
Non-UNIX Timestamping has Me Stamping in Frustration
In the slow process of updating things here on this site I have nearly finished with the restructuring the HTML (exception is the about page, which everytime I start on it the changes I start to make lead quickly to a bigger redesign).
Taking a break from the last HTML page restructuring I was looking at finally getting to correctly timestamping and listing the post times based on blog posting location. Everything after the 1783rd blog post currently picks up the server time stamp and that server is sitting in Eastern Australia, so everything (other than blog posts from Sydney) are not correctly timestamped. Most things are 14 to 15 hours ahead of when they were posted - yes, posted from the future.
Looking at my MySQL tables I didn’t use a Unix timestamp, but a SQL datetime as the core date stored and then split the date and time into separate varables created with parsing timestamp in PHP. This leaves 3 columns to convert. It is a few scripts to write, but not bad, but just a bit of a pain. Also in this change is setting up time conversions that are built into the post location, but shifts in time for day light savings starts adding pain that I don’t want to introduce. I’ve been considering posting in GMT / UTC and on client side showing the posting time with relative user local time with a little JavaScript.
I would like to do this before a server / host move I’ve been considering. At this point I may set up a move and just keep track of the first post on the new server, then at some point correct the time for the roughly 500 posts while the server was in Eastern Australia.
OmniOutliner Counts to Four
One of my favorite applications that a lot of my work and workflow lives in and through, OmniOutliner, updated today. OmniOutliner 4 finally was released today. Its interface becomes a little easier to use for more advanced functions, but if you use the iPad version the new Mac version now looks and works a little more like the iPad version (I think this is a good thing for consistency and ease of use).
I have been using OmniOutliner since version one. I learned to think and organize in outlines and I loved in the old days of WordPerfect the start a document in an outline and then start fleshing it out allowed me to work in the same manner I learned in the fourth grade in Mrs. Norman’s class at Raleigh Park Elementary. This seemed natural to prepare writing this way and once WordPerfect went missing from my workflow other writing tools faked outlines and I looked for good outlining tools to be that foundation. OmniOutliner filled this void. But, once I found OmniOutliner I found other fans who had scripted it to do really helpful tasks, like capture web site maps and dump them into OmniOutliner to annotate and arrange them, then use a script to push into OmniGraffle to visualize. Doing this in 2003 (or so) was pure joy. Not only was was OmniOutliner easy to use, it was really powerful because it was well structured and scriptable.
OmniOutliner is Where I Think
About 2003 I was asked by friend Jesse James Garret, “What tool to you think in?” At that point my answer was OmniOutliner. OmniOutliner was my capture tool that allowed for easy structuring and arranging of order. In years to come with OPML becoming the glue to connect many things in my workflow, I would would move my outlines from OmniOutliner to a mind mapping tool and back and forth. This moving the outline into a mind map allowed me to see it and see relationships spatially and to identify order, modify structure, and make connections between nodes in different branches of the mind map. From the mind map I could take all the modifications and move them back into the outline and tweak a little more. From this point it was moving into writing or into a Keynote presentation (also with a script that would take the OmniOutline and convert it to a presentation to flesh out visually).
The Initial Foundation of What Became OmniFocus
With OmniOutliner I went through the early productivity layer for it that later turned into OmniFocus. My old business started and was kept on schedule in that precursor to OmniFocus that Ethan Schoonover cobbled into and on top of OmniOutliner that was called Kinkless GTD (or KGTD for short).
I still think in OmniOutliner. I have all of the (now) 54 elements of the social lenses tucked in there with their hundreds of sub-nodes. This outline is what became the initial foundation for the four days of walk through of them with Dave Gray for what would turn into the Connected Company book. The collection of similar outlines are all within easy reach. I have a saved Spotlight search in the Finder sidebar that aggregate all my OmniOutliner files for one easy view across everything.
OmniOutliner 4 Offers Even More Potential
I really look forward to how OmniOutliner 4 becomes a new part of my world and workflows. The AppleScript looks robust (I didn’t try it in the many months of beta, but look forward to it now). With scripting and the structure there is a whole lot that is possible.
HTML5 Demo Watch
Thanks to the Berg Friday Links I found the Suit up or Die Magazine and Cut the Rope HTML5 demo sites.
Both have me thinking this is really close, then I remember one of my favorite periodical apps, Financial Times went HTML5 more than a year ago. FT went HTML5 to better manage the multi-platform development process needed for iOS and the multitude of Android versions. While many have said the development is roughly 1.5x what it would take for just one platform development it does same incredible amount of time building an app across all platforms. Since all the major smart phone platforms have their native browsers built on webkit, there is some smart thinking in that approach.
Personally, my big niggle with the FT app is while it is browser based doesn’t have Instapaper built-in and it moves me out of the app to send a link of an article (often to myself because lacking Instapaper) rather than natively in the app, or exposing browser chrome so that I can do that while still remaining in the app and in reading their content mode. It would be really smart for FT to sort this out and fix these as it would keep me in the site and service reading, which I am sure they would love. If they could treat both of those like they do with Twitter and Facebook sharing out all within the app it would be brilliant.
The Data Journalism Handbook is Available
The Data Journalism Handbook is finally available online and soon as the book Data Journalism Handbook - from Amazon or The Data Journalism Handbook - from O’Reilly, which is quite exciting. Why you ask?
In the October of 2010 the Guardian in the UK posted a Data Journalism How To Guide that was fantastic. This was a great resource not only for data journalists, but for anybody who has interest in finding, gathering, assessing, and doing something with the data that is shared found in the world around us. These skill are not the sort of thing that many of us grew up with nor learned in school, nor are taught in most schools today (that is another giant problem). This tutorial taught me a few things that have been of great benefit and filled in gaps I had in my tool bag that was still mostly rusty and built using the tool set I picked up in the mid-90s in grad school in public policy analysis.
In the Fall of 2011 at MozFest in London many data journalist and others of like mind got together to share their knowledge. Out of this gathering was the realization and starting point for the handbook. Journalists are not typically those who have the deep data skills, but if they can learn (not a huge mound to climb) and have it made sensible and relatively easy in bite sized chunk the journalists will be better off.
All of us can benefit from this book in our own hands. Getting to the basics of how gather and think through data and the questions around it, all the way through how to graphically display that data is incredibly beneficial. I know many people who have contributed to this Handbook and think the world of their contributions. Skimming through the version that is one the web I can quickly see this is going to be an essential reference for all, not just journalists, nor bloggers, but for everybody. This could and likely should be the book used in classes in high schools and university information essentials taught first semester first year.
Developing the Web for Whom?
Google Web Developer Toolkit for the Closed Web
Andrew in his post "Reading user interface libraries" brings in elements of yesterday's discussion on The Battle to Build the Personal InfoCloud. Andrew brings up something in his post regarding Google and their Google Web Developer Toolkit (GWT. He points out it is in Java and most of the personal web (or new web) is built in PHP, Ruby [(including Ruby on Rails), Python, and even Perl].
When GWT was launched I was at XTech in Amsterdam and much of the response was confusion as to why it was in Java and not something more widely used. It seems that by choosing Java for developing GWT it is aiming at those behind the firewall. There is still much development on the Intranet done in Java (as well as .Net). This environment needs help integrating rich interaction into their applications. The odd part is many Intranets are also user-experience challenged as well, which is not one of Google's public fortés.
Two Tribes: Inter and Intra
This whole process made me come back to the two differing worlds of Internet and Intranet. On the Internet the web is built largely with Open Source tools for many of the big services (Yahoo, Google, EBay, etc.) and nearly all of the smaller services are Open Source (the cost for hosting is much much lower). The Open Source community is also iterating their solutions insanely fast to build frameworks (Ruby on Rails, etc.) to meet ease of development needs. These sites also build for all operating systems and aim to work in all modern browsers.
On the Intranet the solutions are many times more likely to be Java or .Net as their is "corporate" support for these tools and training is easy to find and there is a phone number to get help from. The development is often for a narrower set of operating systems and browsers, which can be relatively easy to define in a closed environment. The Google solution seems to work well for this environment, but it seems that early reaction to its release in the personal web it fell very flat.
13 Reasons
A posting about Top 13 reasons to CONSIDER the Microsoft platform for Web 2.0 development and its response, "Top 13 reasons NOT to consider the Microsoft platform for Web 2.0 development" [which is on a .Net created site] had me thinking about these institutional solutions (Java and .Net) in an openly developed personal web. The institutional solutions seem like they MUST embrace the open solutions or work seamlessly with them. Take any one of the technical solutions brought up in the Microsoft list (not including Ray Ozzie or Robert Scoble as technical solutions) and think about how it would fit into personal site development or a Web 2.0 developed site. I am not so sure that in the current state of the MS tools they could easily drop in with out converting to the whole suite. Would the Visual .Net include a Python, PHP, Ruby, Ruby On Rails, or Perl plug-in?The Atlas solution is one option in now hundreds of Ajax frameworks. To get use the tools must had more value (not more cost or effort) and embrace what is known (frogs are happy in warm water, but will not enter hot water). Does Atlas work on all browsers? Do I or any Internet facing website developer want to fail for some part of their audience that are using modern browsers?
The Web is Open
The web is about being browser agnostic and OS agnostic. The web makes the OS on the machine irrelevant. The web is about information, media, data, content, and digital objects. The tools that allow us to do things with these elements are increasingly open and web-based and/or personal machine-based.
Build Upon Open Data and Open Access
The web is moving to making the content elements (including the microconent elements) open for use beyond the site. Look at the Amazon Web Services (AWS) and the open APIs in the Yahoo Developer Network. Both of these companies openly ease community access and use of their content and services. This draws people into Amazon and Yahoo media and properties. What programming and scripting languages are required to use these services? Any that the developer wants.. That is right, unlike Google pushing Java to use their solution, Amazon and Yahoo get it, it is up to the developer to use what is best for them. What browsers do the Amazon and Yahoo solutions work in? All browsers.
I have been watching Microsoft Live since I went to Search Champs as they were making sounds that they got it too. The Live Clipboard [TechCrunch review] that Ray Ozzie gave at O'Reilly ETech is being developed in an open community (including Microsoft) for the whole of the web to use. This is being done for use in all browsers, on all operating systems, for all applications, etc. It is open. This seems to show some understanding of the web that Microsoft has not exhibited before. For Microsoft to become relevant, get in the open web game, and stay in the game they must embrace this approach. I am never sure that Google gets this and there are times where I am not sure Yahoo fully gets it either (a "media company" that does not support Mac, which the Mac is comprised of a heavily media-centric community and use and consume media at a much higher rate than the supported community and the Mac community is where many of the trend setters are in the blogging community - just take a look around at SXSW Interactive or most any other web conference these days (even XTech had one third of the users on Mac).
Still an Open Playing Field
There is an open playing field for the company that truly gets it and focusses on the person and their needs. This playing field is behind firewalls on Intranet and out in the open Internet. It is increasingly all one space and it continues to be increasingly open.
My Book Highlights from 2005
I had been hoping to do a longer post on things I enjoyed in 2005, but the year end has been a little busier than planned (it is a good thing). There are lot of books that I picked up in 2005 (possibly in the 30 to 50 range). Two from earlier in the year I really enjoyed, John Thackara's In the Bubble: Designing in a Complex World was a huge favorite. It has brought to mind many changes that we need to address as people who design and build things. The second was Personal, Portable, Pedestrian : Mobile Phones in Japanese Life by Ito, Okabe, Matsuda (editors), which provided insight into the Japanese mobile culture. Some of the cultural aspects to dot translate to the United States, but it does a wonderful job of raising awareness for introspection into one's own culture. The book also does a fantastic job of providing history and technological change and differentiation in use based on product types.
The couple months I have picked up some books that I will really have an impact on me. Shaping Things by Bruce Sterling my be my favorite book of the year (with the Thackera a close second). Shaping things gets to where we are going with design in an innovative way. The design of the book is playful and makes for a quick read (unless it gets your mind wondering in wonderful ways, as it did mine). It is a book I could not more highly recommend.
I finally picked up DOM Scripting: Web Design with JavaScript and the Document Object Model by Jeremy Keith and was pleasantly impressed with the coverage of pages that degrade properly. This book is required reading for anybody doing JavaScript or AJAX so to stop the madness with sites that do not work across browsers or mobile devices. There are far too many sites that have developers that did not learn anything from the mid to late 90s and using JavaScript that are making unusable sites, pages, and applications today. If you are a manager of interface developers this is written in a manner that it should be easy to read and make sure the developers working for you are following these guidelines.
My favorite fun book was recently released book in Britain, Against the Wall by and about Banksy. This is a collections of Banksy's street art and wonderful sense of play. I personally find it a great book to break through conventional thinking and get to the other side. This book is not currently available in America, which is an utter shame.
I may do a posting on music from 2005 I really enjoyed in the next day or so. But on this note, I wish you a wonderful new year and great things.
Make My Link the P-link
Simon hit on plinks as an echo to Tim Bray's comments and variation on Purple Numbers (Purple Numbers as a reference). As I have mentioned before, page numbers fail us and these steps are a good means to move forward.
Simom has also posted in more plinks and in there points to Chris Dent's Big Day for Purple Numbers.
I have been thinking for quite some time about using an id attribute in each paragraph tag that includes the site permalink as well as the paragraph with in that entry. This would look like: <p id="1224p7">. This signifies permanent entry 1224 and paragraph 7 with in that entry. What I had not sorted out was an unobtrusive means of displaying this. I am now thinking about Simon's javascript as a means of doing this. The identifier and plink would be generated by PHP for the paragraph tag, which would be scraped by the javascript to generate the plink.
The downside I see is only making edits at the end of the entry using the "Update" method of providing edits and editorial comments. The other downside is the JavaScript is not usable on all mobile devices, nor was the speed of scrolling down Simon's page that fluid in Safari on my TiBook with 16MB of video RAM.
Saving Safari tabs
OS X Hints offers saving Safari tabs
Using HTML tags properly to help external search results
There are some essentials to building Web pages that get found with external search engines. Understanding the tags in HTML and how they are (rather should be) used is important. The main tags for most popular search engines are the title, heading (h1, h2, etc), paragraph (p), and anchor (a). Different search engines have given some weight in their ranking to metatags, but most do not use them or have decreased their value.
Google gives a lot of weight to the title tag, which is often what shows in the link Google gives its user to click for the entry. In the title tag the wording is important too, as the most specific information should be toward the front. A user searching for news may find a weblog toward the top of the search ahead of CNN, as CNN puts its name ahead of the title of the article. A title should echo the contents of the page as that will help the ranking of the pages, titles that are not repeated can get flagged for removal from search engines.
The headings help echo what is in the title and provide breaking points in the document. Headings not only help the user scan the page easily, but also are used by search engines to ensure the page is what it states it is. The echoing of terms are used to move an entry to the top of the rankings as the mechanical search engines get reinforcement that the information is on target for what its users may be seeking.
The paragraph tags also are used to help reinforce the text within them.
The anchor tags are used for links and this is what the search engines use to scrape and find other Web pages. The text used for the links is used by the search engines to weight their rankings also. If you want users to find information deep in your site put a short clear description between the anchor tags. The W3C standards include the ability to use a title attribute which some search tools also use. The title attribute is also used by some site readers (used by those with visual difficulties and those who want their information read aloud to them, because they may be driving or have their hands otherwise occupied) to replace the information between the anchor tags or to augment that information.
Example
The application I built to manage this weblog section is build to use each of these elements. This often results in high rankings in Google (and relatedly Yahoo), but this is not the intent, I am just a like fussy in that area. It gets to be very odd when my posting weblog posting review of a meal at Ten Penh is at the top or near the top of a Google Ten Penh search. The link for the Ten Penh restaurant is near the bottom of the first page.
Why is the restaurant not the top link? There are a few possible reasons. The restaurant page has its name at "tenpenh" in the title tag, which is very odd or sloppy. The page does not contain a heading tag nor a paragraph tag as the site is built with Flash. The semantic structure in Flash, for those search engines that scrape Flash. Equally the internal page links are not read by a search engine as they are in Flash also. A norm for many sites is having the logo of the site in the upper left corner clickable to the home page of the site, which with the use of the alt attribute in a image tag within an anchor link allow for each page to add value to the home page rant (if the alt attritute would have "Ten Penh Home" for example).
Not only does Flash hinder the scapeing of information the use of JavaScript links wipes out those as means to increase search rankings. Pages with dynamic links that are often believed to ease browsing (which may or may not prove the case depending on the site's users and the site goals in actual user testing) hurt the information in the site for being found by external search engines. JavaScript is not scrapable for links or text written out by JavaScript.
Java and AppleScript intro
Mac OS X Hints offers an Intro to Java that interacts with AppleScript. This intro works with iTunes, but can work with any AppleScript available application.
Mac OS X Hacks proves to be very good
This weekend I picked up Mac OS X Hacks by Rael Dornfest and Kevin Hemenway for O'Reilly Books. I have picked up a few new tricks and have some new shareware to look into. The price is very reasonable, which made it an easy decision to purchase. The book is well written and has been bed time reading and couch reading, which has not worked well for getting too many of the good ideas implemented, but that will come.
One interesting section (there are 100 sections) is a speach recognition section that incorporates Perl, AppleScripting, SOAP, speach recognition, and voice output. This contribution by John Udell was a very juicy tidbit that had me thinking of all the wonderful uses.
Understanding Visual Organization
Luke Wroblewski has a must read article, Visible Narratives: Understanding Visual Organization published at Boxes and Arrows. The article shows the importance of and how to visually structure information to assist the user with finding and focussing on content they are interested in. This lesson is one that is often missed in Web site redesigns.
A visual presentation of information is an essential tool to have in your tool belt. Lack of a usable visual structure can hinder your users from finding the information they are seeking. Many users come to a new site and perform a quick scan of the information available looking for something to attract their attention as it relates to terms, visual cues, or a vocabulary that will get the user to the nuggets they desire.
The user's eye needs resting places to guide them or help the user jump from topic to topic until the user finds one topic or link draws the user (as the user believes) closer to the information. Visual organization help facilitate the user's scanning and reading.
If the visual organization uses HTML markup's header tags and CSS for presentation the information has an underlying structure. The underlying structure can be used to assist bots (non-human search tools that scrape sites looking for information) in finding information. The automated scraping or searching is augmented by the markup as the information in the headers is often given greater value and can help the information get consumed by users interested in finding and using the information. With a little bit of scripting a properly marked-up Web page can generate a table of contents. This visual structuring eases the reuse of information, which is always a benefit.
AppleScripts for Safari
Apple Scripts for Safari is a good place to watch for helpful tools. I am finding there is not a "mail link" in Safari, well that I could find. I may take my first whack at AppleScript to scratch this itch. [hat tip Jason]
Yahoo does PHP
Yahoo presentation on why they are moving to PHP. This would make a great interview of write-up as PowerPoint presentations are largely worthless with out the speaking that accompanies them. This is one presentation that has a tiny bit of information that makes me crave for more. I use PHP here as a scripting language of choice. I love being able to use it at work for many of the reasons outlined by Yahoo. It is better than ColdFusion, ASP, or JSP as far a server requirements, secure, and time to market. The flexability and speed which one can develop is tough to beat, except for the flexability of Perl (there is a reason it is called the duct tape of the Internet). The maxim has been use PHP where you can and Perl where you must. Other languages pale in comparison, but have marketing dollars, which drive the hype. [hat tip Cam and Anil]Perl and ColdFusion updates for OS X
A couple reminders of things to look at, well atleast for myself: Updating to Perl 5.8 on Mac OS X and installing ColdFusion MX on OS X.PHP development with Apple's OS X Developer Tools
Apple provides information to use Apple's Developer Tools (for OS X) to build PHP. This will be a very nice mobile tool.Site map with Apple Script and OmniGraffle
A very nice approach to sitemap generator with Applescript and OmniGraffle. [hat tip Michael]Sounds like know nothing
A word to the wise, if you want to seem like you know what you are talking about it is ASP (ay - ess - pee) not asp (like the reptile). If somebody is paying big bucks you should at least sound like you know what you are talking about. They did pronouce VB properly, but they were actually talking about VBA (we let that one pass). These were "guys in suits" so they must know what they are talking about. Do you think we should ask for a code review? Thought so.5k Contest is live again
Yes, it is that time of the year for the 5k Contest. Yes, 5kb of wholesome goodness with which to work. That is graphics, HTML, scripting, and all the the ones and zeros you can pack in..Net lock in
Eric (glish) Costello brings Chris Laco's comments about .Net to his own site as Chris' comments reflect Eric's comments. The main issue is lock in and severe lack of choice. No the security issue that plagues Microsoft at every turn did not show up. The speed improvement in .Net over the current ASP/VB/C development is noticed and raved about. With security a growing concern on many folks minds building applications with a system that only will run on one operating system, which has the worst security record hands down, is not a great option. There are other options available.Not that I personally need this at the end of the month, but Joao Prado Maia writes about Improving Performance by Profiling PHP Applications (benchmarking PHP) on O'Reilly Net. The article shows us how to place scripts within our scripts that allow us to capture times so to know where we could spend our time focussing on making PHP run more quickly.
Dean points to Mac OS X ad redirecting. I like the approach and the means bethid it.
Those of us trying to develop or debug ColdFusion applications at home can take solice in ColdFusion single user license. This is only available with a registered version of ColdFusion Studio on the same box.
Do you build Web pages? Do you have Mac? Do you have to convert text to HTML/XHTML? If you answered yes (if you didn't you should see what you are missing) please go check out Dean Allen's AppleScript for writing on the Web. These should be wonderful additions to our tool belt.
The current term the neighbor from the Graduate would say is "Web Services". If you have Mac OS X, then O'Reilly Net's "AppleScript Primer for Mac OS X" is your passage to understanding how to do Web Services today. This means you don't have to wait for the expensive .Net Framework as you have the the technology now.
Matt makes observations of the state of severs and scripting deployments. I agree with nearly all of what Matt point out. Some of the reasoning behind the varying set-ups is for security reason's, others are to mirror configurations on other servers that had slightly different purposes. In all, what this needs is a solid documentation tool. PHP provides some of this with a function that prints out the build of the server that script resides on, this is usually the first task many of us perform on a machine. This however, is just the tip of the iceberg of the information we need.
This is part of the second and fourth element of the cornerstones of information application development (info apps need to be usable, maintainable, reliable, and repeatable). If a task is difficult to maintain and even harder to repeat there is some work that needs to be done to change the environment or the application.
Grokdotcom provides a good overview of the methodologies of Web application development. The modular approach is provided as the best methodology, go figure with an open methodology that is based on published/documented standard set of processes. The article ends up discussing Fusebox, which is new to me, but seems to be a rather straightforward approach to modular software development.
I am so happy as I have a system at home that is stable enough to build the Active State components: Komodo, ActivePerl, and ActivePyton. It is nice to have an option to the command line and to be able to test with out FTPing scripts up to a server.
Web-Building provides website development resources. Don't be afraid of the initial presentation, this is a one-stop-shopping resource for HTML, scripting, application development, and everything in between.
ASP2PHP does a solid job handling converting ASP (active server pages) to PHP. The report that Sun sponsored to hopefully show JSP (java server pages) taking over the site scripting market, not only showed JSP had shrunk in market share, but ASP had dropped drastically. The big winner in this survey was Perl with about 40 percent of the market and PHP was ahead of ASP.
Microsoft has given itself problems with ASP this past year or so as the next generation of ASP requires complete rewrites of your ASP code to work on the next version of server. The code is different enough that it will not port, it must be rewritten. This stopped ASP development to a large degree. Some folks have tried running ASP on UNIX with ChiliSoft, but that is only advisable as a patch until you can recode the application, as it is quite resource intensive and you adding another layer of interpretation, which never helps.