RJHarrison.org

Facebook have released HipHop, a piece of software that converts a PHP script into C++ and then compiles it into a static binary. That binary operates it own HTTP server.

About a year ago I wrote a couple of posts about my thoughts on Apache and how we don’t really use it much and actually why I think it’s still relevant. In one fell swoop Facebook has provided an answer/solution to my ruminations; it is now possible (I think) to run PHP as a standalone application server (kinda like Rails with Mongrel, or Python with CherryPy et al).

Most of the applications I write have a single entry point that bootstraps the framework and handles all the routing - my “index.php”. For this scenario, using HPHP sounds like a win: you compile your index.php and instead of using .htaccess to route a set of urls to it, you just configure Apache to reverse-proxy those requests to your backend HPHP instance/s (not sure yet if it will thread, fork or block yet). You could also still run regular php scripts on that Apache server, and just offload some requests to the HPHP daemon.

It sounds like it will really come into its own when you’re essentially running Apache as a container for a monolithic PHP application. Instead of running Apache+PHP on a backend server, you just run the HPHP application. Win! (This is going to rock for API/REST services.)

Some questions:-

• How will it handle autoloading and dynamic classes? Presumably we’d have to give the compiler a list of directories of libraries to include in the compilation?
• What about 5.3? I’ve personally switched, does HPHP support it? (Also, will HPHP and PHP releases be synced?)
• How does the HTTP/server stuff work? Is it a single process that manages worker-pools, does it thread or what?
• How will multiple applications work? I’m guessing each one will need a new HPHP daemon.

The Old

Most developers I’ve encountered run a local LAMP stack (WAMP5, MAMP, etc), but I think they suck:  they abstract away the cool stuff, like compiling extensions, modules and other fun software that you might want to play with. How’s a newbie going to learn some Apache administration if all they do is press a button called “Start Apache”? Some, myself included, run a separate Linux server. The benefits of having your own linux box for development include having an environment closer to production (assuming you deploy on linux), more flexibility in the software that you compile on it, and somewhere to practice your linux-fu. It’s just better.

I use an online box for development (an Amazon EC2 instance specifically) which means it’s accessible anywhere I am, so long as I have a decent internet connection. Others have a machine on their local network, which removes the internet requirement, but introduces a physical constraint. Some crazy fools run linux as a desktop, which is probably not acceptable for most people. Recently the network connection at my office has been crappy and is beyond my control. This has interupted my coding  so I’ve taken the opportunity to freshen things up and switch to using a virtual machine, as well as finally dropping ZDE 5.5.

The New

My new environment is similar to the old: I still use my Mac for writing code and I still run a linux server; the difference is by running the linux server as a virtual machine (via VMWare) I eliminate the network connection. This is the best of both worlds: I write code locally (on my pretty Mac) but also have a linux environment.

VMWare Fusion can be configured to provde your instances with static IPs, which lets you make an entry in /etc/hosts for a domain like devserver.

There isn’t anything special about this setup but I’m very happy with the glue/configuration that holds everything together.

MacFuse: exposing the VM’s filesystem

MacFuse is a cracking piece of kit. It allows you to “mount” a remote filesystem using SSH (or whatever protocol you like). This means my Mac has a path /Volumes/devserver that points to the /home/richard path on devserver. My IDE can now open and edit files on my linux server as if they were local.

I also run a SSH server on devserver and shell into it from my Mac Terminal. The VMWare window is permanently minimized and I rarely interact with the GUI.

Netbeans 6.7 - win!

Finally. I can say farewall to my old friend Zend Studio 5.5 :D. It’s a bit of a shame Zend went down the Eclipse route imho, I liked the old ZDE, but it’s time to move on. I’m not sure entirely what changed since Netbeans 6.5, but whatever it is I like. It took a few hours of fighiting but I beat it into submission  (I really dislike their “Run Configuration” dialog when creating a new project). In Netbeans-speak, my projects are setup as “local server”, but with a “remote” hostname (devserver). The sources directory is set to something like /Volumes/devserver/projects/myproject.

The feature that made it possible for me to switch is the new “Path Mapping” capability for debugging (Project Properties -> Run Configuration -> Advanced…). It maps a remote path (/home/richard/projects/myproject/www/foo.php) into a local path, so that the IDE can open up a local file to display the debugging stuff (ie /Volumes/devserver/projects/myproject/www/foo.php). In my setup, the translated local path is actually the remote file, due to the MacFuse magic. This means Netbeans will display exactly the same file as the debugger on the remote server is working with. Neat trick! (I also had to create a mapping to my devserver’s PHP include_path, which was /usr/local/lib/php => /Volumes/devserver/.phplib, where the .phplib was a symlink to /usr/local/lib/php). (This allows me to debug any PHP libraries that I have installed.)

Apache Magic: mod_vhost_alias

The module mod_vhost_alias allows you to dynamically create new virtualhosts without editing any config files. I have it setup so that a domain in the format XXX.devserver points to the document root /home/richard/projects/XXX/www. So when I create a new project in my editor at /Volumes/devserver/projects/new-project it is immediately available to view, debug or profile at http://new-project.devserver.

Conclusion

I’m happy. This feels like a nice, self-contained environment.

• Develop on a linux server.
• Write code on my Mac (or Windows).
• Not reliant on the network.
• Can do “remote” debugging.
• Netbeans is lightweight and pretty.
• Bonus: Kcachegrind is available to me through the VMWare instance.

Apache (the http server) is one of the major reasons why PHP is so popular; but with today’s trend of using front-controllers have we obsoleted our old friend?

In the olden days we used to write web scripts in Perl and Apache would have to fork a new interpreter process to execute each script request. This was really slow, especialy on the hardware available at the time. PHP came along and provided an embedded interpreter (mod_php) and gave a huge performance boost to the web-scripting world. This was a major motivator for migrating from Perl to PHP. It doesn’t hurt that Apache also happens to be a solid piece of software that can be (and probably has been) be compiled on pretty much all operating systems out there. PHP has unashamedly piggy-backed on its host’s popularity, and for that, we salute you Apache!

Another reason PHP gained popularity is that you could drop a PHP file into any directory and Apache would just run it - no need for the “cgi-bin”. Today, website best-practice would typically recommend against executing “physical .php files” like add-user.php, and instead have pretty urls like /users/add (good for readability and for SEO). Apache provides us with such functionality though a variety of methods, the most popular being mod_rewrite. We can easily use rewrite rules to route /users/add => add-user.php. Thing is, we’re not happy with that. We’ve all got drunk on “MVC” and cool applications use front-controllers (a single entry point for all requests, normally “index.php”). So instead of using Apache’s built-in functionality, we’re re-implementing the wheel inside our own applications. Why use mod_rewrite when we can do the same thing, but slower, using Zend_Controller_Router_Route_Regex? /me rolls-eyes.

We’re even ditching Apache for static stuff too. If you don’t already, it’s probably only a matter of time before you run CSS and Javascript request through PHP too (for good reason). Yahoo’s frontend performance guidelines suggest reducing the number of HTTP requests is important, so it makes sense to generate (and cache) a single CSS file and a single JS file. You might also do clever stuff like postfixing the urls with a version number to help caching, and/or gzip/minify/manipulate headers. These are good things to do, but you’re now using Apache even less.

So what is left? Images, video, flash? Lots of sites which graduate beyond a single-server use a “static file server” to handle this stuff. There’s also no point using a feature-rich, but slower, server like Apache for doing basic static file serving, when alternatives like Lighty/nginx can do this more efficiently. Another option might be to locate static content on a NAS of some sort which Apache will alias (Alias /images /mnt/nas/images). We’ve now got a document root that looks like this:-

index.php
.htaccess (to route everything to index.php)

Wow, clean! (Our PHP code is outside the docroot in some include_path location). I just had a thought - we could get rid of the document root entirely, but sticking this in our virtualhost: “php_value auto_prepend_file /path/to/app/index.php”. (That would be pretty funny, running a site without a docroot).

We don’t really use Apache for much, so why do we use it at all? I think it boils down to being a nice host environment for PHP. There’s nothing really wrong with that, except PHP is tightly coupled to Apache. I think Apache rocks, but if we just end up using it as a host for PHP, then that’s bloated overkill? What about if we stripped out the features of Apache that we dont use until we’ve got a lightweight http wrapper for our PHP app to run in… isn’t that what Rubyists do with Mongrel?

The type of code I write in PHP today is request/response stuff. I get an incoming HTTP request, do some processing, and create an HTTP response. That sounds obvious, but the subtle difference to what I was doing 5 years ago, is that my PHP code is taking care of ALL of the request lifecycle. I don’t use Apache for authentication, logging, uri-routing, headers setting, gzip, caching etc - everything is done by my code. It’s this slight shift in paradigm that makes me think “scripting application servers” like Mongrel+Rails or CherryPy are worth keeping an eye on, and another reason why I question the future of PHP.

I’ve been “a web developer” for about 12 years now (crikey!). The first few were spent with Perl and then I moved over to PHP when I accepted that whilst mod_perl was “better”, it also was also worse. Perl CGI was the defacto language for web scripting but it was slow and cumbersome to write web apps with. Then mod_perl came along and removed the speed issue and gave the developer more power, but it was also much harder to implement, and normally involved two apache instances with one acting as a reverse-proxy for the other, and of course required a dedicated server (when dedicated servers were expensive!). Around the same time PHP 3 was taking off and it gave the speed boost of mod_perl but without the headache of implementation and made it much easier to write web scripts, particularly in a shared hosting environment.

Looking back at that transitional time, I get the vibe that a similar shift is happening today. From Perl->PHP developers got an easy life, with lots of the legwork that you used to do taken care of (like parsing the query string - s/%([a-fA-F0-9]{2})/pack “H2″, $1/eg; #ring any bells?). I see the “new kids” on the block like Python and Ruby offering developers an easier life. They’re offering not only things like Rails/Django-type frameworks, but also a much more attractive package of support tools like gems/eggs, capistrano, migrations and both have unit-testing frameworks built in as a part of their core distributions. Ten years ago the average web developer didn’t care about unit tests, source control and deployment, but now we do and these two alternatives offer something more attractive.

This timing of this blog post and the recent namespace discussion/decision in the PHP world are not coincidental. I’m disappointed with how things have played out with that whole fiasco. The end result is pretty horrible from an average Joe developer’s position (fugly syntax, confusing separator) but the how the community / core devs reached this point is the saddest part. There was so much bickering, trolling and flaming, which to me indicates a much larger problem: something’s wrong with the management of the project. It’s also not just this incident, there was drama recently with “contributor licenses” and PDO2 and I get the vibe that the activity of the project/community decreased a lot in recent years. It wasn’t so long ago that “PHP6″ was going to be the next big thing and unicode was going rock our socks, but now we’re still struggling to get a 5.3 release out the door and suddenly v6 seems farther away than ever. We also have a BDFL who contradicts the direction of the language. Rasmus does quite a bit of hating on “new PHP” style code and promotes an old-school approach (”let’s all use SQL inside our HTML!” ;)). I have no problem with someone having this opinion, but for that person to also be the guiding star of PHP is obviously an issue (I wonder if he wishes BDFL was a position one could resign from :D). Take namespaces as a good example of a lack of leadership - there was no one around to smack the kids, tell them to stfu and to make an executive decision as the mature adult.

The situation with PHP5 also depresses me. I know we can prove anything with statistics… but it’s 4 years old now, PHP4 is official dead/end-of-life and yet PHP5 represents only 40% of installations. IE7 was released only 2 years ago and it looks like is has about a 50/50 split with IE6. Perhaps of greater concern is how many users are actually developing with PHP5? I’d bet that there are more “PHP4″ developers out there than “PHP5″. If so, then does that drop the potential number of developers active in PHP5 open source projects? Back in the early 2000s there were loads of active PHP projects - everyone was on a level playing field and writing code for the same version. Today, I’d struggle to name a popular PHP5 project outside of frameworks: where the hell is our sexy PHP5 version of OSCommerce, “PHPAdsNew”, the classic forums, or even blogs (ok, Habari). From what I can see, it’s just not the same as it used to be.

Times have changed. The web has moved on and so have the needs of its developers. I think we might have reached the peak of PHP. It’s clearly going to be a language that remains popular for years to come, but I think it’s like Perl when PHP 3 was released. It does the job, but there are compelling reasons to switch to another language which is now beyond the “fan boy” stage and is maturing (within the web-app sense). I also think it’s not just a “PHP vrs Python” or “PHP vrs Ruby” question, and also has a lot to do with how web architectures are evolving generally (distributed clouds, REST services, better caching options, prevalence of AJAX).

I’m conflicted about writing this post because the PHP job-market is really good at the moment and PHP is “breaking through into the enterprise”. I think there are more conferences going on around the world than there ever has been and there are lots of other signs that PHP is doing well. I don’t think the sky is falling or anything dramatic, but I do think the wind is changing. Perhaps the global economic situation might also affect us? There’s less capital expenditure going to be made, so does that mean less investment in rewriting old PHP4 systems to PHP5? Maybe it’ll be a good thing for PHP, in that companies won’t be risking/investing in moving away from PHP (and maybe even switching to PHP because it’s cheaper?). Or will it act to speed up the stagnation?

I plan on being a PHPer for years to come but I’ve got a sneaky feeling I’ll be having an affair with another language in the not too distant future. On that note… time to read a little more of my new Python book ;-).

Progress April has been coming along nicely. I’ve written most of the HTTP layer (Client, Request and Response). There are a few “extras” left to code, but essentially I’ve now got a functional “80/20″ HTTP client.

Usage looks like this:-

$request = new April_Request("http://www.google.com/search");
$request->GET['q'] = “REST frameworks”;
$request->GET['hl'] = ‘en’;
$request->setHeader(’user-agent’, ‘Mozilla/5.0′);

$client = new April_Client();
$response = $client->request($request);

if($response->isOk()){
    echo $response->getBody();
}

If you’re familiar with Zend_Http_Client then the API will be familiar (although it’s quite different under-the-hood).

I’ve also started implementing the “REST Framework” part of the project. This part has gone through quite a lot of refactoring recently as I’m not 100% sure how best to implement it (as a design challenge). The primary goals are:-

1. Make it easy for someone to implement a REST API from the client perspective*
2. Respect a “Resource Orientated Architecture” (ROA)

* I’m deferring the “server” aspect for now (I think it’s an area that gets blurry with “standard frameworks” and I don’t want to go there!).

Respecting an ROA basically means restraining from creating RPC-like services (à la Zend Framework). So the service doesn’t expose methods like $s3->ListAllMyBuckets(), but allows you to perform an HTTP-verb action (HEAD/GET/PUT/POST/DELETE) upon a resource: $s3->getBuckets()

I’ve come up with a nifty solution that uses PHP’s magic __call() method. It breaks apart the verb and resource (”get” and “Buckets”) and can then take the appropriate action in order to “get Buckets”. So if the REST service has a resource called Object the service would let you call:-

• getObject(…)
• putObject(…)
• headObject(…)
• postObject(…)
• deleteObject(…)

Internally the service object has an HTTP client, so when one of these methods is called it must:-

1. Build an HTTP request object, based upon the verb, resourceType, resource, url and any other parameters that it needs.
2. Execute the HTTP request and return the Response

How an HTTP Request is built depends mostly on the service (an Amazon S3 request will be different to a Flickr request) and on the Resource (an S3 object-resource is represented differently than a Flickr image).

I’ve reflected this in the class-design: you extend the base April_Service and override the buildRequest() method to apply any service-specfic logic when building a Request. The resource-specific logic is handled slightly differently: we use the Strategy Pattern and load in a ResourceHelper object. The reason for this twofold:-

1. A service might have many resources and it’s easier to maintain the code if their nuances are self-contained (imagine a single My_Service class trying to handle a dozen types of resources).
2. Resource-specific logic is also required when decomposing the Response to our HTTP Request (ie we have an HTTP Response with some XML… how do we convert that to a PHP representation of the resource?).

So the ResourceHelper essentially converts a “resource” to/from PHP and “service” representations.

At the moment I’m really pleased with how this design has played out. In order to implement a particular REST service you just need to create a Service class and then a ResourceHelper for each type of resource.

Here’s some code to demonstrate all this:-

April_Service: April_Service.phps
The base service, extend this to implement a REST API.

April_Paste2: April_Paste2.phps
An service implementation for a Paste2.org API

April_Paste2_Paste_Helper: April_Paste2_Paste_Helper.phps
A ResourceHelper for a “Paste” resource within the Paste2 service

paste2test: paste2test.phps
A simple use-case showing how the Paste2 service could be used

just read a post by Michael Kismal about PHP certification and it hit me that taking the Zend Certification in 2006 was actually one of those pivotal moments in my life - A real “butterfly flaps its wings and causes a storm” thing.

I took the exam back then because I felt my CV was weak (having really just worked on my own projects/startup before then). Shortly afterwards, I got a job with a small UK firm (Llama Digital), where I worked with a developer named Anto Heley. The ZCE was definitely something Llama were interested in and I’ve no doubts that it helped me get the job. Whilst there, I also worked alongside the guys from Sugarcane Studio (who later go on to do my branding for Pluggable).

After 6 months at Llama I got poached by “Zend UK” (who merged with iBuildings.nl last year). They tapped me up through the ZCE Yellow Pages - a direct result of me taking the certification. I moved to London and started going to PHPLondon meets and met a bunch of really cool PHP people, including Paul Morgan who organised this year’s PHPConference.co.uk. Through the job I met and worked with some interesting/high-profile people and clients.

After 6 months at Zend UK (september ‘07) I left to start up Pluggable. Around that time, my old colleague Anto got in touch to say he’d moved jobs and his new employer needed some PHP help, and they became my first Pluggable client; a few more months after that, a company I worked with whilst at Zend UK became my second Pluggable client. These two relationships have since become the biggest % of my work in the last 6 months, and the work for one of them inspired my article for PHP Architect.

At PHPConference.co.uk, I volunteered to help out Paul, and also managed to do a little branding stunt by exchanging elePHPants for business cards, which helped me make me a few more contacts, friends and hopefully clients. I had also suggested the “PHP Framework Throwdown” talk to Paul and he asked me to chair the discussion: that led to me blagging along a spot to the speakers’ dinner and having the pleasure of meeting some “Famous PHP People”.

So looking back over the last two years, all of my career paths, the cool PHP stuff I’ve done and people I’ve met can be traced back to a single causal event: taking the Zend Certification. I know this is a little bit cheesey but I honestly can’t imagine what my life today would be like had I not taken it - and I really like where I’m at right now.

$125 was money well spent

This evening I kicked off coding my new project (codename: April) which at its core is an HTTP Client library for PHP5.

Prior to starting writing code, I’ve done a fair bit of research into the other options that are out there, including analysing their source code to see how they do things. I don’t think I’ve ever looked at 3-4 different libraries side-by-side like this and compared in depth how they do essentially the same thing. It’s very interesting to see the different approaches the authors take as well as the bigger picture of their frameworks.

Of the libraries that I’m using for reference (Zend_Http_Client, Solar_Http, PEAR::HTTP_Request and PECL::HTTP) only Pecl::HTTP fully separates Request/Response/Client. The others take a 2-component approach and have a Response object plus either the Request built into the Client (Zend) or the Client built into the Request (PEAR & Solar). I personally think it’s important to separate each of the three components: the input, the output, and the bit in the middle to gain maximum flexibility and code reuse.

Another area I looked at today was how URLs are handled and validated (or not!). Solar and Zend both have URI objects and PEAR has Net_Url(2). Solar and PEAR both use parse_url(), PEAR validates the parts using regexes but Solar does no validation checking (using whatever parse_url() spits back). The Zend guys clearly don’t trust parse_url and their implementation has lots of regexes to split the URI and validate it. None of them use PHP’s Input Filter, which conveniently has sanitizing and filtering calls for URLs (it could be that it’s for URLs and not URIs).

That reminds me - I spent about an hour of my coding session researching and arguing with myself whether I should be using URL or URI terminology. My conclusion was I prefered URL (an URL specifies a location of the resource, which is implicit in an HTTP request); but also that URI was valid because in the context of an HTTP Client one should assume that the URI was a URL (there’s no way it could be a URN - just doesn’t make sense). Time well spent I think :-).

So, after lots of reading and thinking I wrote some test cases for April_Http_Request and put pen to paper to write her first function: setUrl() (and associated alias, setUri()). It took me 4hrs and it’s only one function but I think I’ve given it my fullest attention! I want this project to use the best bits from my reference libraries and I’m quite happy to spend time learning how they tick. I’m also going to use their tests (both Solar and Zend [and April] use PHPUnit) so my code should at least cover the same “gotchas” as they do. This worked amazingly well for testing my setUrl() method: I managed to reuse almost all the tests for Zend_Uri_Http and after a few failures I tweaked my code to pass - setUrl() is now covered by 25 or so tests (without me having to write them).

Fingers firmly crossed I can continue doing this, as that’s a lot of brainwork that I can reuse (under the New BSD license).