WEB PROGRAMMER

The third of Clarke’s three laws states, in layman’s terms, that anything you cannot understand is basically magic. In our world we are so accustomed to technology that we rarely think about how it works, or how it must look to an outsider.

Any sufficiently advanced technology is indistinguishable from magic.

Arthur C. Clarke

Bringing a modern day cell phone to an impoverished country and telling them that all they have to do is press a few buttons and speak to someone across the world with crystal clear clarity must really be a shock. Imagine looking at our tiny devices, with no wires attached, not even any real, pushable buttons, and then speaking into it to someone else. Religious people might believe they are talking to the heavens. Others might think it is sorcery. Even myself, a person familiar with the workings of cell phones, and radio waves and everything else involved in making a cell phone work, can admit that it is very spectacular to see in action.

So when I say that programming is akin to magic,  it isn’t in the Harry Potter style with wands and sparks and secret communities. It isn’t about whispering a few words and having locks be unlocked, candles get lit and making objects float. Programming is magic because with the right combination of words, environments and knowledge, you can make SOMETHING appear from seemingly NOTHING. Is that not magic? Have there not been countless people persecuted for making something happen that others could not understand?

In witchcraft, a person would take a foreign or made up language, they would light their candles in a pentagram shape and summon demons or whatever it is witches do (I am a programmer, not a witch!). Well in the world of web development we take made up languages, we type them into computers in a specific order with strict formatting and we summon websites.

We keep making our “spells” (AKA code) longer and more complex, we start mixing various languages together and we keep pushing boundaries. The end result being crazy and wonderful websites that can automate, entertain, communicate, simplify (and complicate!) our lives. 

My peers may understand the magic I use, they can probably even use it better. But really, out in the general population, how many people see us as magicians and wizards and witches and warlocks? Many times I have come across people who tell me they simply do not understand how we are able to create such things all from some text on a computer. 

There are other professions, such as carpentry, where people can watch and see a progression from pieces of wood to a beautiful table. We can watch an artist take a blank canvas and create a masterpiece with nothing but paint. But most people looking over my shoulder just see gibberish being typed in and results coming out. Magic!

About the author: Christopher Waldau is an avid fan of street magic. Ever since he was a little boy people like David Blaine would captivate his imagination and fascinate with simple card tricks and illusions. He has also created a portfolio website for a close friends father who practices the art of magic. As such, he may have biased opinions on his views of programming being likened to magic. As he never had the skill and/or dedication to focus on magic, he likes to think they are one and the same. Hopefully there are people out there who he can fascinate and inspire them to choose the path in life that makes them happy.

Not too long ago I decided to re-design and re-launch my personal website. I didn’t know it would coincide with the new year, but with change comes….more change, I guess!

I felt that I outgrew the old design. It was literally the first full site I ever made. And while it was good all those years ago, both myself and the web development world have grown and evolved.

So let’s talk about the new site. First off I used (but heavily modified) a theme from Styleshout. They have amazing themes FOR FREE! They just ask for some love (AKA a link-back) somewhere on the site. To be specific I used the Ceevee theme.

My previous site was technically a “one-page” site, but with some fancy javascript that gave the illusion of different pages. Ceevee is also a one page theme. So naturally, that is the first thing I changed. Don’t ask me why, but I pretty much hate single page sites. Mostly as a design, but also for the programming. Too much scrolling is annoying, and so is not being able to cater each page specifically how you want it made.

So I separated all the pages, gave them each their own containers and their own links. But then I realized I needed a header, a navigation bar and a footer for each page. Want to change the page title, add something to the menu or change the copyright text? Make sure you remember to do it in all the pages! That isn’t very convenient so the next step was converting the HTML site into a PHP site.

Using PHP I was able to create individual files for header, nav and footer and each page now “includes” each of those files. That means that by simply changing the one relevant file, changes will be made across the entire site.

The method I used to create each PHP page was to call the page in the links. So http://chris.waldau.ca/?p=resume would bring up my resume page. I would also be able to use this in PHP to determine what page I am on and dynamically change and adjust certain things, such as the page title (seen in the browser tabs, for example you should see “Blog | Christopher Waldau” in this tab). But having question marks and equals signs are not very user-friendly in a URL so I created a .htaccess file to redirect the “ugly” links back to “pretty” ones.

I wanted my site to be a little more interactive and programming oriented so I found an amazing JQuery plugin called typed.js by Matt Boldt. I used it to make my home/landing page more fun by having my name and an intro blurb get typed out when the page loads.

The original contact page did not include any sort of captcha so I figured it would be best to implement one. But a captcha can be pretty annoying for a user who just wants to enlist my services. As oppose to having a captcha that requires the user to validate themselves, I implemented one that lets robots invalidate themselves. They are called honeypot captcha’s and the idea is that a robot spammer is not smart enough to distinguish a real form field from one we have hidden from view. The robot spammer will read the website source code, ignore the CSS that says a form field is hidden and simply fill out the entire form. When I process the form I check if the honeypot has been filled out or not. If it has been then it was more than probably a robot spammer and I reject the form. Don’t worry though, I still send back a nice response explaining the situation, in case something went wrong and an actual user has seen and filled in the honeypot field.

I also added a blog, since the original Ceevee template did not include one. I used styling cues from the “about” page in the original template so that my blog could still match the rest of the website. This very blog is actually powered by Tumblr. I am using the Tumblr API and a PHP class built by Dhruv Govil (again, something I heavily modified) to interface with Tumblr. Unfortunately I can’t find the original script anymore (my Google-Fu has failed me!) but it is a pretty basic script that requests all the posts from a specified user and displays them. There is no functionality for creating/editing posts through the PHP class itself, so when I want to write a new post I go to Tumblr and write one from there. The advantage being that Tumblr is already set up to be made for blogging and has all sorts of great features that I would otherwise have to build myself. When a new post is made public on my Tumblr blog, it is instantly also available on my personal website.

Other changes to the original Ceevee template include text (obviously!) the landing page background (Montreal represent!) and other CSS changes (mainly the colour scheme). Oh, and I also swapped out the “latest tweets” for “latest blog post” on the contact page since I don’t use Twitter.

All in all I am very pleased with the result as well as my increasing skill with all things web development. Found a bug? Want to make a suggestion? Use my brand new contact page!

P.S. For legacy sake, I have left my old site available at http://chris.waldau.ca/old. Lets all go laugh at it!

I always liked the idea of things like Google Drive. A place where I can store all my documents and photos and invoices and to-do lists that is always accessible from wherever I might be. But there is a very real danger with online cloud storage: you have no control over the machines that are storing your personal info.

Look at something like the now defunct Megaupload. Granted, many people used it as storage for files they may not have technically owned anyways, but it wasn’t solely a piracy haven. Near the end Megaupload was advertising that a bunch of Fortune 500 companies had premium Megaupload accounts. 

What happened to all those documents and files after the raid, and subsequent takedown, of Megaupload? Last I heard, the majority were still in limbo. Some companies who hosted Megaupload servers are still holding on to the servers with their data intact at the request of the United States Government, in order to be used as evidence against Kim Dotcom. I have heard some other companies simply wiped the servers in order to sell them to new clients. Megaupload account holders do not have access to their files anymore.

Sure, one could argue that Google is legit and this would/could not happen to such a large company. But what does Google (or any other cloud hosting service) really do to protect our data and our privacy? Do they keep backups somewhere? Do they snoop our content? Are they vulnerable to hackers who are looking for nude celebrity pictures? What if there is a natural disaster at one of the server facilities?

This is where an important technology rule comes into play: always keep multiple backups! That USB on your keychain can easily be lost, the hard drive on your laptop may get corrupted, the cloud hosting service you trust might shut down and your home may get robbed. Keeping multiple backups in different locations ensures you will still have access to your valuable data in case anything should happen.

I still heavily rely on Google Drive for keeping my documents ready and up to date. But occasionally I like to back up those older, no longer needed documents to a 2TB hard drive I keep at home. So, what if I need easy access to an old word document or a picture? Not much harder than plugging in the hard drive to my laptop and grabbing what I need. 

But like I said earlier, I like the convenience of having it accessible on the go. With that in mind, I set out to repurpose my Raspberry Pi. (Did I forget to mention I replaced it with an OUYA console? A Tegra 3 processor with Android beats the hell out of a Raspberry Pi as an HD media centre.) I envisioned a Google Drive copycat, where I could go to a URL, login, and be able to view and edit my documents. All while having them stored safely in my own home.

To achieve this I, surprisingly, was able to keep the latest Openelec image on the Raspberry Pi. I installed a fresh image, to make sure there were no lingering settings, preferences or files and set up my wifi connection during the set up.

What is great about Openelec is that it will auto-mount any external drives connected to it on boot by default, no need to worry about that. Next step was to assign the Raspberry Pi a static IP from the Openelec settings and then forward the appropriate ports on my router to the now static IP. Allowing your router to connect to the static IP is integral to connecting to the Raspberry Pi from anywhere in the world where you have internet access.

Openelec also has SFTP configured by default and for some use cases this could be all a person wants. But not me! I wanted Google Drive like access remember? I did some searching and found VSFTPD, which I installed to Openelec (how-to via http://matfrapp.blogspot.ca) so that it could open up direct FTP access on port 21.

There were some important bits of info in the comments, so if you are looking to implement this yourself, be sure to read through them! One in particular was necessary to get the whole system running: you will want to CHOWN the /storage/opt/vsftp folder and its contents. Then open vsftpd.conf and set “anonymous_enable” from yes to no. This is also important, unless you want your HDD open to anyone who knows the IP address to connect to. With that being done though, you will need to use the Openelec default username/password combination (usually root/openelec respectively).

Next I installed MonstaFTP on my server. My real server where I host my various websites for myself and clients as well as side projects like this, not the new one I am making with the Raspberry Pi. MonstaFTP is basically an FTP program wrapped up in website form. You can have a copy on your server and use it to connect to any FTP you know the details to. I modified my copy of it to be single purpose, only connecting to my HDD through my Raspberry Pi. So I hacked in the Openelec user/pass into the code and instead of a landing page requesting FTP details, I changed it to a landing page that requests simply a username and password. I made a few other modifications as well, pertaining to how files are viewed and edited. I also added ability for a few extra filetypes that do not come out of the box with MonstaFTP.

Now, I can login to my 2TB HDD from anywhere and browse, upload, download, view and edit all the files stored on there. So easy, I rarely even physically plug in to the HDD, regardless of the fact I am sitting right next to it!

For the past week or so I have been using OpenELEC as the software on my Raspberry Pi. Not really any reason, just was testing it out and had everything configured so I just let it be. Didn’t feel like reinstalling all my addons and entering passwords. It runs quite well and I didn’t have any real complaints.

Until today.

I came home from work to find the USB WiFi adapter I purchased had arrived in the mail. Just so happens that earlier in the day I had done some research into using OpenElec with a WiFi adapter. I came up with mixed results. I was disappointed at the lack of information I could find regarding OpenELEC and the Realtek 8188CU chipset that my adapter would be using. However I was happy to find that the search terms “openelec wifi 8188cu” gave me 4 links to this very blog! Two are direct links to the blog (One at the Tumblr location and the other at my personal domain where the blog is powered using the Tumblr API). The other two are Tumblr search pages for tags of Raspberry Pi. 

Anyways, back to the matter at hand. I found a few forums where people had problems with their adapters and fairly inconclusive solutions. I let it go and figured I would try for myself to see the results. After plugging in the adapter and going into the OpenELEC OS Settings, that comes pre installed with OpenELEC (under programs), I enterted the necessary information for my home network and rebooted the Pi. No internet. Verified everything was entered correctly and I didn’t fat-finger my password, rebooted again and still no internet access. I didn’t particularly feel like jumping through all kinds of hoops to get the adapter working so I decided to try with Raspbmc.

Raspbmc explicitly states that the Realtek 8188CU chipset has “support out of the box”. I had to wipe my SD card, run the Raspbmc installer, boot the Pi and let it do its thing. Once completed (and still using the wired connection) I installed the Network Manager addon, unplugged the ethernet cable and plugged in the WiFi adapter. Rebooted once more and navigated to the Network Manager addon and it auto scans for available Wifi networks. Success! It found my home connection and asked for the password and it is now saved and running smoothly. 

I had to once again download my various addons and enter all passwords and OAuth keys but this time once everything was set I made a backup of my SD card using Win32DiskImager so that if my card corrupts or I decide to try OpenELEC or XBian, I could reinstall my custom image of Raspbmc and be up and running in under 10 minutes with all my preferences saved. 

Note: I have noticed that Raspbmc locks up and essentially the Pi is frozen and needs a reboot. This happened a few times before I switched to OpenELEC and has happened once since reinstalling Raspbmc but never with OpenELEC. I will have to keep an eye out for this to see if I can isolate when it happens and if it will be a problem. 

Clear acrylic case. A more permanent housing solution for the Raspberry Pi.

Today I received my clear acrylic case in the mail! Saw it in the mailbox on my way out to work so I ran back in the house to get my Raspberry Pi so I could put it together while at work!

It looks awesome! All openings fit perfectly, the case snapped together tightly and once assembled seems extremely sturdy. My only worry at this point is that since the HDMI slot is pretty much flush with the edge of the board and the acrylic has a few millimeters of thickness, I am wondering if the HDMI cable will be able to be pushed in far enough to make a proper connection with the board. I will have to check that out and update this post later. However I am confident these cases have been thoroughly tested with multiple boards since this is a UK company manufacturing them and not just sloppily put together in China.

For now this case gets an A+ for manufacturing, ease of assembly, precision fit for the ports and overall cool factor. As a bonus these cases are stack-able for those who have more than one!

UPDATE: The HDMI plugs in without a problem. After a few days of usage the case is holding up well and I am very satisfied with it.

So I managed to get my LG remote working with the Raspberry pi by modifying my remote.xml file in XBMC. I had to remap my Previous/Next buttons on the remote to work as Back/Menu buttons instead. This seemed the most sensible approach since my use of previous/next is mostly limited to rewind/fastforward and this same function can be achieved with the left/right directional arrows on the remote. Essentially the previous/next buttons were useless! Now keep in mind these changes were specific to my particular Lg tv model. It seems newer models dont have the back button issue so normally the remote would work out of the box. 

I also remapped by pause button to bring you all the way back to the home menu, except when your in fullscreen video/music (then it actually pauses the media). Also made the previous button bring up the power menu when in the home screen (since there is no menu to back out of anyways) and the next button brings up the favourites menu when in home screen. These 2 menus are accessible from the home screen anyway if you navigate to the little icons but a button for them makes just that much easier to access.

Last night when I tried to boot up my Raspi it just wouldn’t. Some kind of VFS mounting error? Possibly a result of changing some of the Raspbmc settings to “super” from the “fast” default. I found the response was slow for my taste and that maybe changing the profile would help. I have heard that some people corrupt their SD cards by overclocking the Raspi. Overclocking is when you make the device perform at faster higher speeds then it is intended to. Raspbmc settings should not have caused this, only if the user defines settings that are outside the limits. In any case it seems my SD card was corrupted somehow and so I would have to reinstall it.

I took this opportunity to try out the competition. OpenELEC is an alternative software that can be installed to the SD card to run the Raspi and boot directly into XBMC. Where Raspbmc is a stripped down version of Debian with only the necessary bits to have XBMC running, OpenELEC is a stripped down version of linux with all the bits required to run XBMC. Apples and Microsofts (AKA apples and oranges). Basically the difference between Mac OSX and Windows. They both do the same thing, just in different ways.

OpenELEC requires for you to build the images yourself. This can be rather lengthy and confusing for beginners. I looked at the instructions when I first compared the different options (and subsequently decided to go with Raspbmc due to the easy installation) and thought it was overly complicated. However I have found that you can download prebuilt images from users. Of course you should trust the person who compiled the images! It would seem that Chris Swan is trusted by many in the community and releases daily prebuilt images for OpenELEC. So I downloaded today’s image, used Win32DiskImager which copies the image to your SD card and booted the Pi! All in all about 5 minutes time. Much quicker then Raspbmc.

So OpenELEC booted up no problem, my LG remote works without any setup, add-ons installed and worked and everything seems normal! Honestly I am not sure I would be able to tell the difference between the two if asked. I do however find it a little bit quicker between menus and whatnot.

Another software I could try out is XBian but from forum posts it seems to have a generally negative view in the Raspberry Pi / XBMC community.

I will probably end up reverting back to Raspbmc mostly since it self updates and is easier to manage. It also has a strong lead devlopper who knows what he is doing and constantly implements new features. More importantly though he only implements a new feature once it is ready and this is the kind of developer I like to think I am. Plus having to rely on other users to build images of OpenELEC is not my cup of tea. Yes, I could learn to compile them myself but I am lazy!

The Raspberry Pi doesn’t come with integrated WiFi to keep costs of the board itself as low as possible. It does have an ethernet port but what fun is it having a media centre that needs a cable running from your computer room to the living room? For this reason I have purchased a USB WiFi dongle from ModMyPi. The advantage of course with this particular model is that it is tiny. Keeping the Raspi as small as possible is one of the reasons it makes for an ideal living room accessory.

Apart from size, another point to consider when purchasing a WiFi dongle is how power hungry it is. The Raspi needs a stable power supply and sometimes WiFi dongles can suck more power then they are suppose to, therefore pulling power away from the board and causing it to crash or corrupt your SD card. This dongle is (suppose) to be able to be plugged directly into the Pi without needing an externally powered USB hub.

A third but important factor in a WiFi dongle is what drivers it requires to be able to be used. This one uses the Realtek 8188CU Chipset. This chipset, among a few others, is compatible out of the box with most Raspberry Pi softwares including Raspbmc.

At a 15$ price point (yes 15$ exactly after exchange rates and shipping!) this makes for a pretty impressive wireless solution.

Cost to date: 84.76$

I call it the iPi!

While I wait for my clear acrylic case to be shipped I have been keeping the Raspberry pi in the box it came in. This is (obviously) impractical as I have to disconnect and reconnect wires every time I use it. One thing with the Raspberry Pi is the huge community behind it doing all sorts of crazy things. Cases is one of the most popular mods that people are putting effort into. There are many official and unofficial manufacturers of cases designed specifically for the Raspi, but what about all those random items you have lying around? As I linked to in a previous post, people have made cases out of cigarette boxes and Lego. After some browsing a found a forum post where the OP was thinking about making a case out of the packaging his iPhone came in. That’s when it hit me. I always kept my iPhone boxes because I thought they were beautiful and I always wondered what I could do with them. So I dug up my old iPhone 3GS box and got to cutting it up. About an hour later I was finished (the cardboard is incredibly thick…) and I had a pretty decent looking housing for my media centre.

Bonus: you can use the piece of plastic that the iPhone comes in as a lid for the box, with the hole in the middle serving as airflow to prevent overheating!

First boot of Raspbmc

Next step is installing a software to the SD card I purchased so that the Raspi can actually boot. I decided to go with Raspbmc since it seems to be the most fully functional XBMC solution for the Raspberry Pi. Raspbmc does not have a stable release yet, so there are still only release candidates which get updated quite often. The most current is RC5 which is what I installed. The process is to flash your SD card, insert it to your Raspi and power on. The rest of the process is automated, it downloads the necessary files, then installs and reboots into XBMC. My first time around was unsuccessful, possibly due to the fact I was plugged in with a composite cable instead of HDMI. RC5 seems to have an issue with composite cable and no image is shown. So I reformatted the SD card, plugged the pi in with HDMI instead and let the the files download and install and it worked! The Raspberry Pi booted into XBMC and my LG remote worked without any further tinkering! At this point the pi is basically very much useable, so after installing the 1channel.ch add-on I took a break and watched an episode of Scrubs. 

I have found a bug however! With certain LG remotes there is no “back” button. Just my luck! So this makes the pi a little less useable at this point as I had to plug in a keyboard to be able to back out to the main menu again. Some research shows there is a way to map a new button to act as the back button. Requires I SSH into the pi and edit a file called remote.xml. Now I run into a problem with SSH! For some reason I am not able to gain access to the pi from another computer. 

Another problem for another day! Happy to see the Raspberry Pi up and running regardless of my remote issues.

So a Raspberry Pi is cheap yes, but still it isn’t a buy and turn on product. The Raspi still needs at minimum a power supply and an SD card of at least 2GB. And then you still need a screen and cable to the screen. I already have an LCD TV (LG) and an HDMI cable (yes the Raspi has HDMI out and is capable of 1080p!) and I managed to dig up an old blackberry charger that is listed as compatible with the Raspi. I didn’t have an SD card available however so I had to go out and buy this Sandisk Ultra 4GB SDHC for 10.29$ tax in.

The Raspberry Pi also comes “naked” all components exposed and for every day use can be fragile. I have seen some people open up their TV’s to install the Raspi, I’ve seen mounts to attach them to the rear of a TV and various enclosures. You can basically make your own case out of anything, including lego and cigarette boxes. I chose to buy a clear acrylic case to show off the awesomeness of the Raspi while still protecting it. Coming from the UK though so delivery delay of 5-7 days according to Royal Mails website. This came to 9.47$ after shipping and exchange rates.

Cost so far: 69.76$

The board really is the size of a credit card!

Today I picked up a Raspberry Pi Model B. They sell for 35$ plus taxes, shipping, customs and exchange rates and most people say they paid around 47$ when it was all set and done. Current wait times are about 4 weeks before they can be shipped out due to high demand and low(ish) production. I found a seller on Kijiji for 50$ and available for pick-up immediately. I didn’t mind to pay the 3$ extra. I plan on turning the Raspi into my main media centre PC, to replace my Macbook (which is replacing my  ~5 year old windows desktop that likes to blue screen whenever it pleases). I have my Macbook running XBMC and it works perfectly, but I feel bad having it run such a mundane task when it is capable of so much more. So if this project completes as expected my Macbook can replace my desktop as my main machine and leave the Raspi permanently hooked up to the living room TV with its sole purpose being to stream content from various sources and controllable from the sofa with XBMC.

Cost to date: 50$