Off the Top: Servers Entries
Site Shift has Started
After seven to eight years with the same web host a move is underway again. This will be slower and it may hit reverse for parts. Having read Phil Gyford’s recounting his host move I think I am prepared for the scope of what this move entails.
In my time with this host I put up dev projects that then moved to their own servers, I experimented with a lot of different tools and services that I continue to use (or would love to get back to using), and I have a boat load of mail accounts set up for contextual use and family.
The are a few domains, a lot of subdomains, many databases, and many applications (my hand built and third party creations). I’m expecting to have to tweek some code and scripting to handle different versions of PHP and Python than I currently run.
Two Host Mambo
The move is splitting things to two different hosts (one a VPS and the other a modern hosted web host). The split is to give some of the more resource heavy applications like Fever and ThinkUp room to flex and grow, these are applications that only I use and are really helpful for a variety of reasons. Other applications I have run locally on a Mac Mini will move there as will old services I shuttered due to lack of web resources availalbe.
The bulk of what is here at vanderwal.net and publicly available will move to a web host with good management consoles and mail (for now). While I am comfortable living and working at command line, I like a good console to make things easier. Also having someone else doing security updates, patching, and upgrades is nice as well so I can focus on the things I really want to do.
Mail is something that I really want to improve. Having to pester to have somebody give the mailserver a kick to get mail functioning again is something that shouldn’t need to happen. I have run my own mail server in the past (10 years back) and I know not to do that again as it was a pain and that pain is much worse with dealing with SPAM as well as getting on the wrong side of a SPAM blacklist (difficult to get off of a blacklist or greylist for domain or IP block when you are fewer than a few thousand accounts). The days of small email services died 5 to 10 years ago and that gets worse with each passing year.
Mail for the moment is going with my web hosting. This is less than optimal, I know that well. But, a good sized web host has the bulk to deal with black / grey listing to some degree. I know my optimal choice is FastMail, but making that move will take more decisions to narrow down mail accounts and sorting the size I am willing to limit some accounts to.
Getting Real
This only gets real when the domains get repointed and propigate. That is not today.
Changing Hosts and Server Locations
I’ve been in the midst of thinking through a web host / server move for vanderwal.net for a while. I started running a personal site in 1995 and was running it under vanderwal.net since 1997. During this time it has gone through six of 7 different hosts. The blog has been on three different hosts and on the same host since January 2006.
I’ve been wanting better email hosting, I want SSH access back, more current updates to: OS; scripting for PHP, Ruby, and Python; MySQL; and other smaller elements. A lot has changed in the last two to three years in web and server hosting.
The current shift is the 4th generation that started with simple web page hosting with limited scripting options, but often had some SSH and command line access to run cron jobs. The second was usually had a few scripting options and database to run light CMS or other dynamic pages, but the hosting didn’t give you access to anything below the web directory (problematic when trying to set your credentials for login out of the web directory, running more than one version of a site (dev, production, etc.), and essential includes that for security are best left out of the web directory). The second generation we often lost SSH and command line as those coming in lacked skills to work at the command line and could cripple a server with ease with a minor accident. The third has been more robust hosting with proper web directory set up and access to sub directories, having multiple scripting resources, having SSH and command line back (usually after proven competence), having control of setting up your own databases at will, setting up your own subdomains at will, and more. The third generation was often still hosting many sites on one server and a run away script or site getting hammered with traffic impacted the whole server. These hosts also often didn’t have the RAM to run current generations of tools (such as Drupal which can be a resource hog if not using command line tools like drush that thankfully made Drupal easier to configure in tight constraints from 2006 forward).
Today’s Options
Today we have a fourth generation of web host that replicates upgraded services like your own private server or virtual private server, but at lighter web hosting prices. I’ve been watching Digital Ocean for a few months and a couple months back I figured for $5 per month it was worth giving it a shot for some experiments and quick modeling of ideas. Digital Ocean starts with 512 MB or RAM, 20GB of SSD space (yes, your read that right, SSD hard drive), and 1TB of transfer. The setup is essentially a virtual private server, which makes experimentation easier and safer (if you mess up you only kill your own work not the work of others - to fix it wipe and rebuild quickly if it is that bad). Digital Ocean also lets you setup your server as you wish in about a minute of creation time with OS, scripting, and database options there for your choosing.
In recently Marco Arment has written up the lay of the land for hosting options from his perspective, which is a great overview. I’ve also been following Phil Gyford’s change of web hosting and like Phil I am dealing with a few domains and projects. I began looking at WebFaction and am liking what is there too. WebFaction adds in email into the equation and 100GB of storage on RAID 10 storage. Like Digital Ocean it has full shell scripting and a wide array of tools to select from to add to your server. This likely would be a good replacement for my core web existence here at vanderwal.net and its related services. WebFaction provides some good management interfaces and smoothing some of the rough edges.
There are two big considerations in all of this: 1) Email; 2) Server location.
Email is a huge pain point for me. It should be relatively bullet proof (as it was years ago). To get bullet proof email the options boil down to going to a dedicated mail service like exchange or something like FastMail, a hosted Exchange server, or Google Apps. Having to pester the mail host to kick a server isn’t really acceptable and that has been a big reason I am considering moving my hosting. Also sitting on servers that get their IP address in blocks of blacklisted email servers (or potentially blacklisted) makes things really painful as well. I have ruled out Exchange as an option due to cost, many open scripts I rely on don’t play well with Exchange, and the price related to having someone maintain it.
Google Apps is an option, but my needs for all the other pieces that Google Apps offers aren’t requirements. I am looking at about 10 email addresses with one massive account in that set along with 2 or 3 other domains with one or two email accounts that are left open to catch the stray emails that drift in to those (often highly important). The cost of Google for this adds up quickly, even with using of aliases. I think having one of my light traffic domains on Google Apps would be good, the price of that and access to Google Apps to have access to for experimentation (Google Apps always arise in business conversations as a reference).
FastMail pricing is yearly and I know a lot of people who have been using it for years and rave about it. Having my one heavy traffic email there, as well as tucking the smaller accounts with lower traffic hosted there would be a great setup. Keeping email separate from hosting give uptime as well. FastMail is also testing calendar hosting with CalDAV, which is really interesting as well (I ran a CalDAV server for a while and it was really helpful and rather easy to manage, but like all things calendar it comes with goofy headaches, often related timezone and that bloody day light savings time, that I prefer others to deal with).
Last option is bundled email with web hosting. This has long been my experience. This is mostly a good solution, but rarely great. Dealing with many domains and multitudes of accounts email bundled with web hosting is a decent option. Mail hosting is rarely a deep strength of a web hosting company and often it is these providers that you have to pester to kick the mail server to get your mail flowing again (not only my experience, but darned near everybody I know has this problem and it should never work this way). I am wondering with the benefits of relatively inexpensive mail hosting bundled into web hosting is worth the pain.
I am likely to split my mail hosting across different solutions (the multiple web hosts and email hosts would still be less than my relatively low all in one web hosting I currently have).
Server Location
I have had web hosting in the US, UK, and now Australia and at a high level, I really don’t care where the the servers are located as the internet is mostly fast and self healing, so location and performance is a negligible distance for me (working with live shell scripting to a point that is nearly at the opposite side of the globe is rather mind blowing in how instantaneous this internet is).
My considerations related to where in the globe the servers are hosted comes down to local law (or lack of laws that are enforced). Sites sitting on European hosts require cookie notifications. The pull down / take down laws in countries are rather different. As a person with USA citizenship paperwork and hosting elsewhere, the laws that apply and how get goofy. The revelations of USA spying on its own people and servers has me not so keen to host in the US again, not that I ever have had anything that has come close to running afoul of laws or could ever be misconstrued as something that should draw attention. I have no idea what the laws are in Australia, which has been a bit of a concern for a while, but the host also has had servers in the US as well.
My options seem to be US, Singapore, UK, Netherlands, and Nordic based hosting. Nearly all the hosting options for web, applications, and mail provide options for location (the non-US options have grown like wildfire in the post Edward Snowden era). Location isn’t a deciding point, but it is something I will think through. I chose Australia as the host had great highly recommended hosting that has lived up to that for that generation of hosting options. It didn’t matter where the server was hosted eight years ago as the laws and implications were rather flat. Today the laws and implications are far less flat, so it will require some thinking through.
Down with DreamHost? Then Up with SegPub
After the security problems with DreamHost in the past day or so (many sites getting hacked as DreamHost is storing their passwords to accounts in open text files (something that should never be done)) I am getting a lot of requests from friends for options.
Look at SebPub for Hosting
Nearly 2 years ago I was getting fed up with my old hosting solution as the outages were unbearable. I needed solid open source and open standard hosting. I found some of my friends were hosted at SegPub and were loving it. After an introduction to SegPub I moved my hosting and have never been happier.
SebPub is an Australian company, which means you pay in Australian Dollars making the price relatively reasonable. The support has been wonderful, often on IM. Occasionally I will get an IM from them asking how things are going. They have kept the small town feeling with personal touch even as they have grown.
Needless to say the personal attention is nice, but the incredibly rare need to talk to them about downtime, problems, etc. is really a nice change. I can not recommend SegPub highly enough.
New Hosting and Peace of Mind
Most of the e-mail coming in is now coming through the new hosting company. I am getting feedback from people that they can see this new site. The change of the domain to the new hosting started resolving correctly in 30 minutes for me, which was very fast.
The choice in hosting came down to two strongly recommended companies, Pair.com and Segment Publishing (SegPub). Both had nothing, but positive recommendations and each had more than four recommendation (Pair actually had over 10 recommendations). Both offered the services I wanted and needed (PHP, Python, secure mail, MySQL, solid up-time records, and Rails (Rails was more of a nice than a need)).
I chose SebPub based on two things that separated it from Pair. One was price, SegPub was a little less and offered a good introduction and trial period. It is an Australian company (servers in the US) and payments are in Australian dollars. Secondly, the first person to recommend SebPub put me in contact with the guy who runs SegPub in a chat. I was able to asks a lot of questions specific to my needs and he was able to easily point me to the needed information or just provide it in the chat. That personal touch was an incredible sell. Just knowing there is that access helped. I had really grown tired of waiting a few hours between responses to trouble tickets and e-mail exchanges and not getting resolution and just getting the ticket closed.
Getting things prepared and moved over took the usual amount of time, but I had far fewer bumps than I thought I would have had. I needed to better optimize MySQL to perform better in MySQL4. I was to the point I needed to make the hosting change and the time disruption was not welcome, but I knew I could focus when the transition was over and done.
So far I have been incredibly happy. I also know that Pair would also be a very good company to go to as well. This option gives me great peace of mind.
A Better Day and Brigher Future
Things are a little better on this end today. I was able to delete the Photoshop French version and reinstall a Photoshop demo (I was very surprised I was able to do this and get back to the exact days left on the tryout where is left of yesterday) so I could continue to work. The shipment of the new package will be the fourth attempt to get this right by Adobe (their stock price is what?).
I have received many great suggestions on hosting and am looking at two of them seriously. E-mail has been up all day today as host hosting, which is good.
Along this front I am really getting tired of my own blogging tool. I no longer have time to keep running and the effort it takes to write, check, and post does not work for me any longer. I am doing too many things at once and not paying enough attention to the actual writing, which I really need to. I also blog adding all the mark-up needed (including needed character encodings). I really want to turn on comments again as readership here has grown and I really want to get back to "conversations" (not just monologues plus e-mail). There are things I want to build that I think would help the blogging community, but it is really fruitless to do this for a tool that has an install base of 1.
I have my options narrowed down for me as I will be running two sites from it and be using it as a CMS as well as a blogging tool. The two candidates I am down to are Movable Type and Drupal. I am leaning toward MT mostly because they have a very active support system inside the company and user-base. Drupal has a killer user-base that is very innovative and the tool has many social and community aspects to it that I really like. I will most likely be playing with both. Both have a good track record running more than one site off an install and using shared components for the different sites.
Now it is sorting out, which I can dump my current site into most easily and clean out the mark-up and encoding so to let the blogging tool do it (will make for easier current specification RSS/Atom feeds). I am also wanting to keep the 1770-plus URLs the same (as well as RESTful), which I have not sorted out. Suggestions are welcome.
Tips on IIS 6
For those of serving the Web from a Microsoft Server 2003 O'Reilly Net has Tips on IIS 6.
Choose your candidate by their Web server
In an effort to get in to the swing of things political, what is your candidate running Web site on? The up time of the RNC and DNC is very telling.
Apache 2.0 builds for OS X
Hmmm... Yesterday I was looking at Apache and the other two pieces of the triumvirate PHP and MySQL. Today I ran across what could be an improved option, Server Logistics' Apache 2.0 along with Perl, PHP, MySQL, and Postgres. Apache 2.0 provides multithreading and other improvements to the incredibly stable and supreme Web server All of this is set to build and run on Mac OX 10.Mac server on a laptop
I did not go to the OS X 10.2 Launch last night (I was having a wonderful time at Fred and Paula's birthday gathering), but I may be picking up Jaguar shortly. When I do so I will revert back to the base Apache build in OS X. After this it will on to O"Reilly Net's Setting up a Site Server article and Marc Liyange pages. It seems when I upgraded the base Apache build on OS X the PHP builds from Liyange were not running properly as they are based on the default builds from OS X.Microsoft embraces Apache Web server
CNet News discusses Microsoft's .Net set to link to Apache, which is a great step as the Microsoft IIS web server is increasingly being dropped as a viable option because of never ending security problems. This would literally doom Microsoft's .Net initiative as it would not be usable on the Internet without their Microsoft Internet server. By moving the ability to run the .Net framework on an Apache server Microsoft not only extends their ability to run their services on a superior Web server with far fewer security problems, but Apache is now recognized as a viable Web server by Microsoft. Apache owns the majority share of the Web server business and those of use that have had the ability to use it prefer it hands down to Microsoft's IIS.Baking versus frying CMS
Aaron discusses baking versus frying with content management and updates bake and fry CMS ideas. The idea is to bake content, which is using your content management system to produce static pages. The alternative is to fry from the CMS by providing truely dynamic content. There are a few reasons why one should choose the frying method:
- Frequent (hourly or semi-daily) updates of informaiton
- Multiple dependancies (information linking to and from many points)
- Unlimited resources
- Many variation of presentation of the data
- Providing user slicing and dicing of informaiton capabilities
- Many external content providers
This list does not capture everything, but also provides maleable guidelines. There are many advantages to baking (publishing static content pages) from a CMS:
- Speed of delivery
- Archievable version
- Ease of troubleshooting and maintenance
- Editable output pages
- Use templates to generate valid mark-up and perfect 508 compliant pages
- Using reusable content pieces that provide consistancy and accuracy of information on all presentation layers
- Keeping various application elements well maintained
Aaron provides good links for further discovery of your own.
IIS on XP Home
Those who have XP Home and want to run IIS (perhaps to build and test ASP) should read 15 seconds IIS on XP Home hack. This is an inexpensive solution to Microsoft not stating anywhere in their marketing materials for XP Home, or in their early tech papers, there is no official way to run an IIS server on the box.Apple means business
Apple announces Xserve that, as Doc writes it, seems to kick the snot out of Windows servers. Faster, more stable, based on an enterprise proven OS, not a virus friendly OS, not a hack-worthy OS, and less money. You do the math.MS security causes sad day
Life sucks when: You have to pull an e-mail account that you manage from service. Particularly when this account is for your Dad. My Dad can be reached at Tom and I will be keeping Thomas. The TJV account is closed.
Why you ask? The account was hacked with the klez virus. He cleaned his hard drive, as he had no choice it or another virus took the hard drive out. He took another hard drive and put it in that machine and started fresh. This may have also infected his new laptop. Yes, all of these machines run Windows (the swiss cheese security system). My dad is more than computer savvy and Windows is not a consumer OS, as it is nothing more than an e-mail away from destroying everything digital you own (among many other issues, which I spend hours assisting friends and relatives with their continual problems with the MS OS). Microsoft continues to lie about its focus on security and the basic problem is the OS itself, it is not secure and it seems it will never be secure. UNIX has some issues, but has many more years of development under its belt, which is why is far more secure. UNIX variants (Apple Mac OS X, Linux, BSD, etc.) all have the advantage of years of experience and advanced developers working on the OS.
Keeping a MS box secure requires somebody with a lot of experience and they are not cheap. The MS total cost of ownership being lower than UNIX is a myth and unfounded. If you have MS open to the outside world (Internet server, DSL at home, or unfiltered (through virus scanner) e-mail, etc.) you need an MS security expert focussed on ensuring the sanctity of whatever is considered valuable on the MS boxes. This person will cost as much, if not more, than a senior UNIX systems administrator (who are, by and large, veterans in UNIX security also as it comes with the territory).
Too many folks (that are near and dear to me) have had MS servers hacked or been victims of viruses in the past couple of weeks. Granted the MS boxes hacked may not have been watched over by MS security experts, but that is what it takes.
Making choices, as far as what language to develop Internet applications, should keep in mind lock in factors. A UNIX only or a Microsoft only solution that requires the application be only run on a certain type of server has never been a great idea. This becomes even more apparent now. In my opinion this has never been a good option. Fortunately, there are many more options available that run on nearly all OS platforms. These include: Perl, PHP, Java (JSP), Python, ColdFusion, etc. Each of these languages have their own plusses and minuses, but if a certain OS platform becomes an unavailable option the applications can relatively easily be moved to another OS. This is not the case with ASP, and even less so the .Net framework (as noted before. Sure ASP can use ChiliSoft, but that is a very short term solution (as you know if you have ever had to use it, it buys you time to recode everything into a portable application language) and requires double to triple the hardware resources to run it compared to ASP on MS or any other language running natively.
All of this is just the beginning of the reasons why I most likely have bought my last Windows machine. The other reasons fall into the areas of trust and pricing. This explanation may follow soon.