Scrape IMDB MyMovies with PHP

pimp.pngI have been wanting to somehow mention the movies I have recently seen and what I thought of them on this blog but was not sure how to best do it.
For a while I was running my own custom code to add movie data and ratings and to display them but this was too cumbersome.
Recently I discovered that the movie rating system at IMDB.com keeps track of all the movies you have ever rated (IMDB.com is by the way an amazing resource for movie lovers and I hope that you have used it before).
Since IDMB is the oracle of movies a better approach seemed to be to continue reviewing and discussing movies on there and attempt to mash the data into my blog.
A few regular expressions later and a Pick IMDB MyMovies with PHP (PIMP) script was born.
The script scrapes a given IMDB MyMovies list and provides the results in a handy (two dimensional) array. It is up to the user to then write the display code to make it fit in with a web page (blog).
A basic file based cache has been implemented to save on the amount of hits on IMDB.
At the core of the script is the following regular expression:
/<a href=\"\/title\/([^\/]*)\/([^>]*)>([^<]*)<\/a> \(([0-9]*[\/I]*)\)( \(.\))?<\/td>([^<])<td align=\"center\" bgcolor=\"\#ffffff\">([0-9]{1,2})<\/td><td align=\"center\" bgcolor=\"\#ffffff\"> ([0-9]\.[0-9])?/i
Be aware that if the layout of the IMDB page changes, PIMP will fail. Of course I will be quick to update the regular expression since I am using PIMP my self.
There are some rumours that IMDB will introduce an open API that will allow developers to retrieve all kinds of movie information. Until that is done, crude HTML scraping techniques will have to do.
Feel free to suggest regular expression or code optimisations.
Download PIMP v1.2 and let me know what you think!
Usage:
——
0. Create an account with IMDB and mark a MyMovies list as public
1. To be able to use a cache file create a directory which is writable by the script process (may need chmod 666). For security reasons this directory should be OUTSIDE of the area accessible from the web.
2. Configure the script with your details (list id, cache directory and etc)
3. Upload the myImdbMovies.php file to your web server
4. Access the script directly or better yet use it as an include
Update 26/06/2006
Added a raw HTTP method if file_get_contents doesn’t work for you (hosting provider restriction.
-1 cache time out will skip caching altogether and avoid possible cache file permission problems.
-1 for listItems will result in all items being returned

Web &gt2.0 Conference, London, UK

webgt2con.pngThis is an early notice that later this Summer I will be organising a Web &gt2.0 Conference in London, UK (exact location TBA).
The attendance fee will be $2.795 ($2 and 795 cents) and all proceedings will be donated to an open source project TBA.
At the conference will be several speakers presenting why Web 2.0 is so yesterday and what Web 2.1 can offer.
There will also be brainstorming sessions to come up with new and tantalising buzzwords – ideally trademark-able after general adoption.

On a more serious note. I am an IT professional and have always favoured O’Reilly technology books when making new purchases. I have often recommended them to friends and colleagues.
The latest attempt by O’Reilly to trade mark Web 2.0 and going after the little man with a Cease and Desist letter have made me think twice.
It could be a sign that O’Reilly have lost touch with their audience and also that they don’t fully understand the term that they were a part of popularising.
How can a company that recently so strongly advocated against the Amazon one-click purchase patent make a such an u-turn I cannot comprehend.
O’Reilly should have been clear from the beginning (2003) and stated their intentions by adding a (TM) every time they used Web 2.0. Had the general public been aware of the pending TM I doubt that the term would have gained such a popularity.
One decent way for O’Reilly out of this mess would be to accept that the term is by now in Public Domain.
As long as O’Reilly is very knowledgeable in upcoming web technologies and a leading force in advocating Web 2.0, no imitators will be able to steal their thunder!

Pixmania shop review

★&#x2606&#x2606&#x2606&#x2606
The good: Cheap prices
The bad: French specifications, no local returns, questionable warranty
The ugly: lousy communications, price changes when you add item to your basket
The title of the post is a bit misleading as I did not actually manage to shop at Pixmania.co.uk. I tried my best but the shop was not competent enough to accept my money.
Pixmania.co.uk is an online shop that specialises in cheap electronics, especially digital cameras.
I opted to pick up my items from a local pick-up shop as this seemed very convenient. I was also hoping to save the £10 delivery charge but instead there was a £10 service charge on customer pick-up orders.
The items arrived promptly at the pick-up store but that is where my problems started.
The staff in the pick-up store was not able to charge my debit card as it involved getting an authorisation code from their bank.
It was obvious that the staff person was inconvenienced that he was required to call their bank. He was then a bit abusive on the phone. I was told repeatedly that there was an error at my bank.
The following communication with Pixmania was very difficult. Their call center is located in France and they were not familiar with the local pick-up shop procedures nor about standard UK debit card usage.
The automated email which is signed by James Felix (Customer Service Director) announces to use the contact address [email protected]. This address bounces however and the only way to send email to Pixmania.co.uk is to navigate through their FAQ (frequently asked questions) and hopefully finding an appropriate previous question.
The company operates from France and all electronic items are according to French standard. This means that instruction manuals and electric plugs are French by default. Pixmania calls this “European specification”.
The local pick-up shop I visited was handing out UK 3-pin adaptors and instructions on how to download the manual in English from the manufacturer’s web site.
For me the main reason for buying an item in the UK instead from abroad at a cheaper price is the assurance of an UK warranty. Due to the French operation of Pixmania I am not sure whether you can take a faulty item to the manufacturer in the UK or whether you have to deal with the manufacturer in France.
Pixmania states that certain faulty items will be dealt with by the manufacturer; others will have to be sent back to Pixmania in France.
An additional oddity was the way Pixmania prices change in real time. Wen I originally tried to add my item to the basket, I was returned an error (in French). Temporary problems are quite common on web sites so I just navigated back to the product. This was 10 minutes later and the price had gone up by £12.
After failing to pay for my item in the pick-up shop, I was told to put through a second order online and pay directly online. Again, the product I was interested in had gone up by £9 and was not a bargain anymore it had once been.
This bad experience of shopping at Pixmania is really my fault. I should have read some of the Pixmania.com reviews at the Review Center before making my decision.
An overall value of 1.2 out of 10 after 141 reviews is a really poor score. Only 6% of the reviewers would recommend to use Pixmania.com.

Google talk

google talkThe Google empire has released yet another (Beta) service which this time is an instant messenger network and client Google Talk.
While their standard is to release ground shaking services (GMail, Google Earth, desktop search and more) this time I have yet to feel the rumble.
Google talk is a lean instant messaging (IM) client that supports high quality voice chats. It also notifies the user of any new Gail messages and there is a short cut to send Gail emails. But that’s it!
You have to have a Gail account to be able to use Google Talk. I have plenty invitations to spare so leave a comment if you need one.
Once again the client is Windows only but since Google talk is “built to support industry standards” you can use compatible clients on other operating systems:
OSX – iChat
Linux – GAIM
Windows – Trillian Pro
The “industry standards” is the Jabber/XMPP protocol and it let’s other clients play with Google Talk network in an authorised way.
This one subtle detail that most users will not notice nor understand makes Google Talk special. It is a stab at the established messengers (MSN, AIM, Yahoo, ICQ) to show them how to play fair with Internet users.
Rumours are many in the blogosphere regarding the potential and future of Google Talk. Will it be the preferred client on the future Google Network? Will it allow searching all chat history similar to GMail search? Will it record all voice conversations, and allow searching? What will be the revenue stream(s)?
I just hope that the original Google motto “Don’t be evil-TM” will not be replaced by the new corporate “Must Make Money” motto.

Easter eggs:
Add [email protected] to your buddy list and send it a text message of “play” to play the old school game Hunt the Wumpus.
Create bold text by using * and italics by using underscore (_)

Free international calls with voip

voip(Updated again, see end of article.)
Voice over IP (voip), also called internet telephony is becoming a stronger and stronger competition to the traditional Telco companies.
Skype has become very popular because it let’s people make free calls between computers. With additional services one can even call land lines all across the globe for a fraction of traditional Telco costs. One annoying issue with Skype is that it is using a proprietary format, ignoring the established SIP protocol.
Voipbuster is a similar application provided by the Swiss Telco Finarea SA. The company has multiple cheap telephony services operating in UK and Europe and some other names they go by in the UK are 1899.com and call18866.co.uk.
Voipbuster is however their first computer client. It has a minimal amount of features but it let’s you make calls and store contacts.
Currently the software is in beta status and during this period Voipbuster offers you free calls to landlines in over 20 countries.
See the official list with rates to see the currently free countries.
To avoid a 1 minute cut-off limit you have to credit your account with the symbolic sum of €1 (70p, $1.20) €5 €10. This is to prevent abuse of the service and since you can pay with Paypal it is just a minor nuisance.
#PAGEBREAK#
The quality of the call is exceptional and always higher than what I get through my standard phone.
The reason being that traditional telephony solutions (PSTN) are often band limited, as much as 300-3400 Hz. Voip is able to utilise a wideband codec solution that uses much higher sampling rate (16kHz) and a upper band of 8kHz.
Back to the SIP protocol. Since Voipbuster supports SIP, their service can be used with any of the many, many available SIP clients (also called soft phones). Some are commercial, some are free. All offer more functionality than the Voipbuster client but not always better voice quality.
I managed to locate a free SIP client that supports conference calls between multiple SIP callers (X-pro). This means I can use Voipbuster service, call to multiple landlines in the free countries and connect them all. End result is a free call for everybody.
#PAGEBREAK#
If you cannot stand calling from your computer, you can even call your self at home, switch on conference and continue the free call from your home phone.
Download X-pro from braintel.net.pk/configuration.htm. Install, start up and hit Control-G to initiate an upgrade.
Set up X-pro to be used with the Voipbuster SIP gateway:

Enabled: YES
Display Name: [VoipBuster user name]
Username: [VoipBuster user name]
Authorization User: [VoipBuster user name]
Password: [VoipBuster password]
Domain/Realm: voipbuster.com
SIP Proxy: sip.voipbuster.com

Disable the lower quality GSM codec by clicking on it.
Dial a number on line 1. Dial a second number or your own home phone on line 2. Hit the conference button and all three parties are now enjoying a free international telephone call.
If you are looking for a catch there is none. If you instead think that free is not a sustainable business model you are very right.
Voipbuster is currently in BETA mode and once any network issues have been dealt with and/or the usage has reached a critical mass I expect similar pricing as on the other Finarea SA products.
One can always hope that competition in Voip has reached such a level that networks are ready to provide flat fee or even free calls to certain international destinations!
PS Currently evaluating the Pocket PC application Stanaphone together with the VoipBuster gateway.
Update November 8th, 2005
#PAGEBREAK#
VoipBuster have now dropped the majority of the free countries. It is now obvious that the 25 free countries was too good to be true.
I was hoping that Voipbuster was about to start a revolution in VOIP where certain common destinations would always be free and income would be generated from other destinations.
Instead it seems that it was a marketing trick to attract new customers and all their 5 Euros 🙁
This wiki page has more in depth information about Finarea and their companies.
There is a new method of making international calls for free as long as you have unused mobile phone minutes:
The VOIP operator Rebtel has a system where both you and your friend (in a foreign country) call a local number and get connected.(more details).
I have confirmed that this works to Sweden, Brazil, USA and likely all other countries that Rebtel support.
Happy calling!

Credence: thwarting p2p pollution?

Certain peer to peer networks suffer from a high ratio of pollution: files that are fake, damaged, infected with viruses or similar. First attempts to lower the pollution were simple voting systems where users voted on whether a specific file was good or bad.
I haven’t had much faith in such voting systems because it seemed too easy to spam the systems with fake votes. Surely enough, various companies have already been identified who pollute the peer to peer networks with fake files and rate the files as good.
Credence is an academic project that has a different approach to the problem: it implements a voter correlation scheme.
Users that vote similarly become grouped together and results on a particular object (e.g. a p2p file) are then correlated by the voting group you belong to.
This means that malicious users will quickly form their own voting communities and have little effect on groups of honest voters.
There is a second benefit from this strategy. Votes from malicious users will not be disregarded but will ultimately start being counted as direct opposites. If a dishonest voter votes positively on an object, the honest user will count that as a negative vote for that object. The dishonest voting circles will not even be aware of what the current status of an object is to other voting communities.
One possible attack on Credence would be by a user that votes honestly on a large amount of certain files but votes dishonestly on one or a few specific target files. It is however very complex trying to analyze how many honest votes need to be placed for each dishonest vote while retaining it’s own credence.
Another common vulnerability of voting systems is the possibility for a single node to assume many different identities (also known as The Sybil Attack). Credence attempts to minimise the risk for a Sybil attack by requiring a large download for each new identity.
For honest users it will not be a problem to download one single large file. For dishonest individuals it will be very costly to download a large amount of files in order to set up fake identities.
Now if it only was possible to combine this robust voting system with the anonymous P2P networking protocol I wrote about earlier. This would make it much more reliable and risk free to share let’s say political opinions..

Google Maps and Google Local finally local to UK

Google have extended their brilliant Maps and Local services to the UK. Until now I have only been using it to see how good online services can be. Now I can use it daily to find addresses, get directions and find services (plumbing especially) in London and UK.
Google Maps is a very intuitive way to present maps. Easy zooming, powerful search and smart dHTML like moving the map around are my favourite features. Wonder if Google plans to add satellite images like they do with the US maps.
Google Local combines Google maps with a local directory of services, restaurants, theaters and much much more. Until now I have been using UpMyStreet and it will be interesting to see how well Google Local performs compared with them.
googlemobile.gifLast but not least, see if Google Local Mobile works with your mobile. It should if it supports XHTML (i.e. not older than 2004).
Having access to free maps, directions and local directory from your mobile is just priceless.
PS The SonyEricsson T610/T630 has problems with the mobile version. Firefox has problems as well, use Internet Explorer if you want to try it from your desktop.
Currently the Yell footer has a bug (omitted quotes in href) which creates a XML parsing error and only sloppy IE can display the results. Stay tuned, should not take long to fix this.

Update May 10th
The error has been fixed. I doubt it was due to the email I sent to Google support (I did get a reply though) but instead due to the official launch of Google Local UK Mobile (GLUM). Have a go!

GMail invites

Ok, I never thought I’d be saying this but I am.
I have GMail invites to share in case you *still* aren’t on GMail.
It’s a very good service with 1GB of storage space. If the so called privacy issues do not scare you away, let me know.
Leave a comment with your email address and a reason why *you* need a GMail account 🙂
d.
Cool GMail hack: GMailFS – GMail file system
Update 08/09/04
Another clever Gmail hack – Gallina, a blog engine based on Gmail

Wireless network available

I don’t understand.
I have the wireless card disabled on my laptop yet when I turn the laptop on, first thing it does is to notify me that a wireless network is in the vicinity and that I have to enable wireless to be able to connect to it.
Here I was thinking that switching off wireless guaranteed that no WiFi signals would enter nor exit my laptop; that I was WiFi invisible.

Mozilla Firefox 0.9 released

Of course I downloaded it as soon as I got the news that a new version of the fabulous browser Firefox was out. I have had just minor issues with version 0.8; sites that are IE “optimised” and forgetting cookies being the major ones.
The new version does not bring any radical changes. Instead it focuses on ease of install and usage of extensions.
Downloading it is fast because the whole application is just 4.5MB.
There is a new dedicated extensions and themes manager that keeps track of what has been added to Firefox and whether a new version is available. Very user friendly.
First thing you want to do after installing Firefox 0.9 is to get new extensions because 0.8 extensions are not re-installed. I managed to corrupt my installation by downloading extensions for the 0.8 version. Go to update.mozilla.org to get the correct extensions and themes!
Google toolbar, IE view, Web Developer and AdBlock are a must.
Update 17/06/04
And if you still are looking for reasons to go away from IE, read Lockergnome’s excellent article on Why You Should Dump Internet Explorer