Programming is magic

The third of Clarke’s three laws states, in layman’s terms, that anything you cannot understand is basically magic. In our world we are so accustomed to technology that we rarely think about how it works, or how it must look to an outsider.

Any sufficiently advanced technology is indistinguishable from magic.

Arthur C. Clarke

Bringing a modern day cell phone to an impoverished country and telling them that all they have to do is press a few buttons and speak to someone across the world with crystal clear clarity must really be a shock. Imagine looking at our tiny devices, with no wires attached, not even any real, pushable buttons, and then speaking into it to someone else. Religious people might believe they are talking to the heavens. Others might think it is sorcery. Even myself, a person familiar with the workings of cell phones, and radio waves and everything else involved in making a cell phone work, can admit that it is very spectacular to see in action.

So when I say that programming is akin to magic,  it isn’t in the Harry Potter style with wands and sparks and secret communities. It isn’t about whispering a few words and having locks be unlocked, candles get lit and making objects float. Programming is magic because with the right combination of words, environments and knowledge, you can make SOMETHING appear from seemingly NOTHING. Is that not magic? Have there not been countless people persecuted for making something happen that others could not understand?

In witchcraft, a person would take a foreign or made up language, they would light their candles in a pentagram shape and summon demons or whatever it is witches do (I am a programmer, not a witch!). Well in the world of web development we take made up languages, we type them into computers in a specific order with strict formatting and we summon websites.

We keep making our “spells” (AKA code) longer and more complex, we start mixing various languages together and we keep pushing boundaries. The end result being crazy and wonderful websites that can automate, entertain, communicate, simplify (and complicate!) our lives. 

My peers may understand the magic I use, they can probably even use it better. But really, out in the general population, how many people see us as magicians and wizards and witches and warlocks? Many times I have come across people who tell me they simply do not understand how we are able to create such things all from some text on a computer. 

There are other professions, such as carpentry, where people can watch and see a progression from pieces of wood to a beautiful table. We can watch an artist take a blank canvas and create a masterpiece with nothing but paint. But most people looking over my shoulder just see gibberish being typed in and results coming out. Magic!

About the author: Christopher Waldau is an avid fan of street magic. Ever since he was a little boy people like David Blaine would captivate his imagination and fascinate with simple card tricks and illusions. He has also created a portfolio website for a close friends father who practices the art of magic. As such, he may have biased opinions on his views of programming being likened to magic. As he never had the skill and/or dedication to focus on magic, he likes to think they are one and the same. Hopefully there are people out there who he can fascinate and inspire them to choose the path in life that makes them happy.

                                                                                                       

A new year, a new site!

Not too long ago I decided to re-design and re-launch my personal website. I didn’t know it would coincide with the new year, but with change comes….more change, I guess!

I felt that I outgrew the old design. It was literally the first full site I ever made. And while it was good all those years ago, both myself and the web development world have grown and evolved.

So let’s talk about the new site. First off I used (but heavily modified) a theme from Styleshout. They have amazing themes FOR FREE! They just ask for some love (AKA a link-back) somewhere on the site. To be specific I used the Ceevee theme.

My previous site was technically a “one-page” site, but with some fancy javascript that gave the illusion of different pages. Ceevee is also a one page theme. So naturally, that is the first thing I changed. Don’t ask me why, but I pretty much hate single page sites. Mostly as a design, but also for the programming. Too much scrolling is annoying, and so is not being able to cater each page specifically how you want it made.

So I separated all the pages, gave them each their own containers and their own links. But then I realized I needed a header, a navigation bar and a footer for each page. Want to change the page title, add something to the menu or change the copyright text? Make sure you remember to do it in all the pages! That isn’t very convenient so the next step was converting the HTML site into a PHP site.

Using PHP I was able to create individual files for header, nav and footer and each page now “includes” each of those files. That means that by simply changing the one relevant file, changes will be made across the entire site.

The method I used to create each PHP page was to call the page in the links. So http://chris.waldau.ca/?p=resume would bring up my resume page. I would also be able to use this in PHP to determine what page I am on and dynamically change and adjust certain things, such as the page title (seen in the browser tabs, for example you should see “Blog | Christopher Waldau” in this tab). But having question marks and equals signs are not very user-friendly in a URL so I created a .htaccess file to redirect the “ugly” links back to “pretty” ones.

I wanted my site to be a little more interactive and programming oriented so I found an amazing JQuery plugin called typed.js by Matt Boldt. I used it to make my home/landing page more fun by having my name and an intro blurb get typed out when the page loads.

The original contact page did not include any sort of captcha so I figured it would be best to implement one. But a captcha can be pretty annoying for a user who just wants to enlist my services. As oppose to having a captcha that requires the user to validate themselves, I implemented one that lets robots invalidate themselves. They are called honeypot captcha’s and the idea is that a robot spammer is not smart enough to distinguish a real form field from one we have hidden from view. The robot spammer will read the website source code, ignore the CSS that says a form field is hidden and simply fill out the entire form. When I process the form I check if the honeypot has been filled out or not. If it has been then it was more than probably a robot spammer and I reject the form. Don’t worry though, I still send back a nice response explaining the situation, in case something went wrong and an actual user has seen and filled in the honeypot field.

I also added a blog, since the original Ceevee template did not include one. I used styling cues from the “about” page in the original template so that my blog could still match the rest of the website. This very blog is actually powered by Tumblr. I am using the Tumblr API and a PHP class built by Dhruv Govil (again, something I heavily modified) to interface with Tumblr. Unfortunately I can’t find the original script anymore (my Google-Fu has failed me!) but it is a pretty basic script that requests all the posts from a specified user and displays them. There is no functionality for creating/editing posts through the PHP class itself, so when I want to write a new post I go to Tumblr and write one from there. The advantage being that Tumblr is already set up to be made for blogging and has all sorts of great features that I would otherwise have to build myself. When a new post is made public on my Tumblr blog, it is instantly also available on my personal website.

Other changes to the original Ceevee template include text (obviously!) the landing page background (Montreal represent!) and other CSS changes (mainly the colour scheme). Oh, and I also swapped out the “latest tweets” for “latest blog post” on the contact page since I don’t use Twitter.

All in all I am very pleased with the result as well as my increasing skill with all things web development. Found a bug? Want to make a suggestion? Use my brand new contact page!

P.S. For legacy sake, I have left my old site available at http://chris.waldau.ca/old. Lets all go laugh at it!

                                                                                                       

Outside access to an external hard drive by FTP

I always liked the idea of things like Google Drive. A place where I can store all my documents and photos and invoices and to-do lists that is always accessible from wherever I might be. But there is a very real danger with online cloud storage: you have no control over the machines that are storing your personal info.

Look at something like the now defunct Megaupload. Granted, many people used it as storage for files they may not have technically owned anyways, but it wasn’t solely a piracy haven. Near the end Megaupload was advertising that a bunch of Fortune 500 companies had premium Megaupload accounts. 

What happened to all those documents and files after the raid, and subsequent takedown, of Megaupload? Last I heard, the majority were still in limbo. Some companies who hosted Megaupload servers are still holding on to the servers with their data intact at the request of the United States Government, in order to be used as evidence against Kim Dotcom. I have heard some other companies simply wiped the servers in order to sell them to new clients. Megaupload account holders do not have access to their files anymore.

Sure, one could argue that Google is legit and this would/could not happen to such a large company. But what does Google (or any other cloud hosting service) really do to protect our data and our privacy? Do they keep backups somewhere? Do they snoop our content? Are they vulnerable to hackers who are looking for nude celebrity pictures? What if there is a natural disaster at one of the server facilities?

This is where an important technology rule comes into play: always keep multiple backups! That USB on your keychain can easily be lost, the hard drive on your laptop may get corrupted, the cloud hosting service you trust might shut down and your home may get robbed. Keeping multiple backups in different locations ensures you will still have access to your valuable data in case anything should happen.

I still heavily rely on Google Drive for keeping my documents ready and up to date. But occasionally I like to back up those older, no longer needed documents to a 2TB hard drive I keep at home. So, what if I need easy access to an old word document or a picture? Not much harder than plugging in the hard drive to my laptop and grabbing what I need. 

But like I said earlier, I like the convenience of having it accessible on the go. With that in mind, I set out to repurpose my Raspberry Pi. (Did I forget to mention I replaced it with an OUYA console? A Tegra 3 processor with Android beats the hell out of a Raspberry Pi as an HD media centre.) I envisioned a Google Drive copycat, where I could go to a URL, login, and be able to view and edit my documents. All while having them stored safely in my own home.

To achieve this I, surprisingly, was able to keep the latest Openelec image on the Raspberry Pi. I installed a fresh image, to make sure there were no lingering settings, preferences or files and set up my wifi connection during the set up.

What is great about Openelec is that it will auto-mount any external drives connected to it on boot by default, no need to worry about that. Next step was to assign the Raspberry Pi a static IP from the Openelec settings and then forward the appropriate ports on my router to the now static IP. Allowing your router to connect to the static IP is integral to connecting to the Raspberry Pi from anywhere in the world where you have internet access.

Openelec also has SFTP configured by default and for some use cases this could be all a person wants. But not me! I wanted Google Drive like access remember? I did some searching and found VSFTPD, which I installed to Openelec (how-to via http://matfrapp.blogspot.ca) so that it could open up direct FTP access on port 21.

There were some important bits of info in the comments, so if you are looking to implement this yourself, be sure to read through them! One in particular was necessary to get the whole system running: you will want to CHOWN the /storage/opt/vsftp folder and its contents. Then open vsftpd.conf and set “anonymous_enable” from yes to no. This is also important, unless you want your HDD open to anyone who knows the IP address to connect to. With that being done though, you will need to use the Openelec default username/password combination (usually root/openelec respectively).

Next I installed MonstaFTP on my server. My real server where I host my various websites for myself and clients as well as side projects like this, not the new one I am making with the Raspberry Pi. MonstaFTP is basically an FTP program wrapped up in website form. You can have a copy on your server and use it to connect to any FTP you know the details to. I modified my copy of it to be single purpose, only connecting to my HDD through my Raspberry Pi. So I hacked in the Openelec user/pass into the code and instead of a landing page requesting FTP details, I changed it to a landing page that requests simply a username and password. I made a few other modifications as well, pertaining to how files are viewed and edited. I also added ability for a few extra filetypes that do not come out of the box with MonstaFTP.

Now, I can login to my 2TB HDD from anywhere and browse, upload, download, view and edit all the files stored on there. So easy, I rarely even physically plug in to the HDD, regardless of the fact I am sitting right next to it!

                                                                                                       

The WiFi

For the past week or so I have been using OpenELEC as the software on my Raspberry Pi. Not really any reason, just was testing it out and had everything configured so I just let it be. Didn’t feel like reinstalling all my addons and entering passwords. It runs quite well and I didn’t have any real complaints.

Until today.

I came home from work to find the USB WiFi adapter I purchased had arrived in the mail. Just so happens that earlier in the day I had done some research into using OpenElec with a WiFi adapter. I came up with mixed results. I was disappointed at the lack of information I could find regarding OpenELEC and the Realtek 8188CU chipset that my adapter would be using. However I was happy to find that the search terms “openelec wifi 8188cu” gave me 4 links to this very blog! Two are direct links to the blog (One at the Tumblr location and the other at my personal domain where the blog is powered using the Tumblr API). The other two are Tumblr search pages for tags of Raspberry Pi. 

Anyways, back to the matter at hand. I found a few forums where people had problems with their adapters and fairly inconclusive solutions. I let it go and figured I would try for myself to see the results. After plugging in the adapter and going into the OpenELEC OS Settings, that comes pre installed with OpenELEC (under programs), I enterted the necessary information for my home network and rebooted the Pi. No internet. Verified everything was entered correctly and I didn’t fat-finger my password, rebooted again and still no internet access. I didn’t particularly feel like jumping through all kinds of hoops to get the adapter working so I decided to try with Raspbmc.

Raspbmc explicitly states that the Realtek 8188CU chipset has “support out of the box”. I had to wipe my SD card, run the Raspbmc installer, boot the Pi and let it do its thing. Once completed (and still using the wired connection) I installed the Network Manager addon, unplugged the ethernet cable and plugged in the WiFi adapter. Rebooted once more and navigated to the Network Manager addon and it auto scans for available Wifi networks. Success! It found my home connection and asked for the password and it is now saved and running smoothly. 

I had to once again download my various addons and enter all passwords and OAuth keys but this time once everything was set I made a backup of my SD card using Win32DiskImager so that if my card corrupts or I decide to try OpenELEC or XBian, I could reinstall my custom image of Raspbmc and be up and running in under 10 minutes with all my preferences saved. 

Note: I have noticed that Raspbmc locks up and essentially the Pi is frozen and needs a reboot. This happened a few times before I switched to OpenELEC and has happened once since reinstalling Raspbmc but never with OpenELEC. I will have to keep an eye out for this to see if I can isolate when it happens and if it will be a problem. 

                                                                                                       

                                                               

Clear acrylic case. A more permanent housing solution for the Raspberry Pi.