May 06 2009

The coding challenge is over. I’ve proven that It is possible to record the streams of Hulu and 4oD if you are determined enough and use the right open source tools.
I am now officially dropping support and removing the following Programme Type plug-ins:

  • Channel4’s 4oD Catch-up
  • Channel Five’s Demand Five
  • Hulu.com

I like a challenge but at this point I’d rather concentrate on the BBC iPlayer TV/Radio and BBC Podcasts. I’d like to reduce the workload and headache that these other sites cause me! Channel4 – you can stop feeding us that ‘Countdown’ episode now (very good!), and Hulu.com – if it weren’t for Boxee I guess you could stop encrypting the HTML that you serve to stop people scraping your web pages and the massive spaggetti of actionscript decryption functions! I guess many users who want to watch Hulu.com and 4oD programmes will now end up using Bittorrent instead. This is really not quite as ideal for these channels as before, at least with get_iplayer, they could control the geography of where their users were streaming from.

I’m still going to keep the ITV Player plug-in supported as most non-windows users will not have the ability to run Silverlight.

If you’ve downloaded a recent version of get_iplayer you’ll notice that I’ve moved it to a plug-in based architecture. This allows users to write 3rd-party plug-ins. Hopefully this will prove useful if you don’t want to rely on me to add your local channels to get_iplayer. I will attempt to write some documentation on writing your own plug-in. I’ve supplied a plug-in example template with lots of comments to get you started here. You can see the remaining plug-ins in get_iplayer SVN (see the download page) to get some idea of how they work.

36 Comments

  • On May 6th, 2009, tvAddict said:

    Shame… but understandable.
    Silly to keep chasing your tail on this!
    It was useful at the weekend to get a few shows from fiver (that I don’t get on Freesat).
    Maybe if they left some adverts in, they wouldn’t lose so much revenue..

    Thanks for the ongoing support

    • On May 6th, 2009, james said:

      4od worked ace for me, i want my old version back! where can i get it from

      • On May 6th, 2009, james said:

        anyone got old version thy wanna send me?

        • On May 6th, 2009, Andy said:

          Blimey, I wondered how long the fast-paced release schedule could be maintained! Every time I looked there was a new version! ;o)

          Don’t blame you. BBC iPlayer has far better content quality anyway, and the plugin system will allow people to support other services if required. I just hope that if any plugins are developed that the metadata layout in the cache files is adhered to; makes coding things against get_iplayer so much easier.

          • On May 6th, 2009, linuxcentre said:

            The cachefile format is not defined in the plugin. That is in the core fo get_iplayer. Generally all you need for a plugin is a programe metadata downloader, page scraper/feed parser and a stream URL extractor. The rest is done in get_iplayer. I’m happy to assist anyone with the plugin stuff as far as I am able.

            • On May 6th, 2009, Andy said:

              Top stuff. Must admit to not having had chance to peek at the plugin code, so that’s nice to know.

              Oh, and thanks for a great piece of software!

          • On May 6th, 2009, james said:

            would i need a plugin then to get 4od back? forgive my lack of understanding but only discovered this today

            • On May 6th, 2009, Strawp said:

              You would need to *write* the plugin yourself.

              The old code has been blocked by C4.

              • On May 6th, 2009, bob said:

                Blocked how?
                Is there some kind of forum we could start sharing information on the plugins?

                • On May 6th, 2009, james said:

                  shame cause was gonna stick stuff on my ipod

            • On May 6th, 2009, Bigarules said:

              Huge thanks from me for all your efforts. I’m quite happy with just BBC support.

              • On May 6th, 2009, andrew said:

                Sad to see Channel 4 and five downloading dropped already. As a viewer in an area with poor TV reception this was the only way I could watch these channels (I could not get their websites to work on my non-Microsoft computer). I would even pay a small fee to allow me to download them! Back to bittorrent I guess.

                I hope someone takes up the challenge to continue supporting the channels and I hope the plugins or links to them are posted here.

                • On May 6th, 2009, ufufgnaf said:

                  It is quite sad to see you drop the support. Could you upload the last version of the plug-ins to somewhere please? Could you also provide links to the 3rd party plug-ins please?

                  • On May 6th, 2009, ufufgnaf said:

                    Thanks for what you have done so far. The scripts are tremendously helpful.

                  • On May 7th, 2009, AudioBear said:

                    It’s a shame Phil, just when I’d got my pvr automated to download programmes and then add them to iTunes with the correct metatdata and naming… mind still no luck with automatically converting ITV… My little Media mac will never be quite the same…. sniff…

                    On the other hand I understand your dislike of receiving millions of emails about Countdown downloads and the perpetual patches.

                    Still a great script though!

                    • On May 8th, 2009, LiLo said:

                      It’s strange that the “countdown” problem still existed. I’m using a patched version of get_iplayer 1.75 and that works correctly for me.
                      Channel 4 and Five both work fine.

                      The Second part of the patch (see below) only adds the Accept */* to sub request_url_retry.

                      If I remember correctly yhe first part of the patch is basically the difference between 1.75 and 1.75

                      This is my patch for version 1.75:

                      — usr/bin/get_iplayer 2009-05-02 01:03:48.000000000 +0200
                      +++ get_iplayer 2009-05-08 10:43:03.000000000 +0200
                      @@ -1618,6 +1618,12 @@
                      # Gets the contents of a URL and retries if it fails, returns ” if no page could be retrieved
                      # Usage = request_url_retry(, , , , []);
                      sub request_url_retry {
                      +
                      + my %OPTS = @LWP::Protocol::http::EXTRA_SOCK_OPTS;
                      + $OPTS{SendTE} = 0;
                      + @LWP::Protocol::http::EXTRA_SOCK_OPTS = %OPTS;
                      +
                      +
                      my ($ua, $url, $retries, $succeedmsg, $failmsg) = @_;
                      my $res;

                      @@ -1627,10 +1633,16 @@
                      return ”;
                      }

                      + my $jph = new HTTP::Headers(
                      +# ‘User-Agent’ => $user_agent{coremedia},
                      + ‘Accept’ => ‘*/*’,
                      +# ‘Range’ => “bytes=${start}-${end}”,
                      + );
                      +
                      my $i;
                      logger “INFO: Getting page $url\n” if $opt->{verbose};
                      for ($i = 0; $i request( HTTP::Request->new( GET => $url ) );
                      + $res = $ua->request( HTTP::Request->new( ‘GET’ ,$url, $jph ) );
                      if ( ! $res->is_success ) {
                      logger $failmsg;
                      } else {

                      • On May 10th, 2009, TG said:

                        When I try

                        get_iplayer –type=’itv’

                        It complains:

                        ERROR: Invalid type ‘itv’ specified. Valid types are: tv,radio

                        I’m running version 1.84. Is ITV support still in get_iplayer?

                        • On May 10th, 2009, linuxcentre said:

                          It seesm that when v1.84 ran for the first time, it got the list of plugins, downloaded get_iplayer again and then failed to download the plugins – did you see any errors on the console?
                          To fix run ‘get_iplayer –update –force’

                          • On May 10th, 2009, TG said:

                            That fixed it – thanks very much!

                        • On May 10th, 2009, keswick said:

                          Hi
                          I know get_iplayer no longer supports 4oD but, is any one else having the same problem when trying to watch a program from ch4 catch up service? Every program, no matter what it is streams as a countdown program but with the title of the program you want…really frustrating…can any one help?

                          thanks

                          • On May 11th, 2009, DDewson said:

                            I am getting the same problem here

                            • On May 12th, 2009, murble said:

                              The trick seems to be, to get a different IP address. Channel4 appear to have a blacklist of IP addresses that have used get_iplayer in the past.

                              • On May 13th, 2009, Matt said:

                                Would they actually block people from using the service properly too though? Would they block UK people at all? Could it be a flash problem causing the Countdown problem?

                                • On May 20th, 2009, Tom said:

                                  My girlfriend’s computer has suddenly started streaming countdown every time like you describe – I don’t understand why it started happening about a week ago as she’s never used anything other than Firefox to access 4oD. Do you have a solution? She can’t get a new IP address.

                                  • On May 20th, 2009, linuxcentre said:

                                    Have you ever used get_iplayer with the ch4 plugin via the IP same address? If so then this will be the reason. Channel4 are blacklisting. If you haven’t then it very much looks like Channel4 may have been rather over-zealous in their blacklisting :-)

                                    • On May 20th, 2009, Tom said:

                                      No, I’d never heard of get_iplayer until I googled the (frankly bizarre) problem and found this page. And my girlfriend is an average Windows Vista user who just wants to watch Gok’s Fashion Fix!
                                      I guess this Countdown episode is Ch4’s idea of a joke?
                                      Oh well, I suppose the thing to do would be to email the 4oD people and get them to sort it out…

                                    • On May 20th, 2009, Matt said:

                                      If you do email them, please keep us updated on what happens.

                          • On May 20th, 2009, AudioBear said:

                            Seems IP address maybe the answer…
                            If I were to use a VPN based on foreign soil, it will work, but for how long :-)

                            • On May 23rd, 2009, Francis said:

                              I have tried to find get_iplayer 1.82, the last before Hulu support was dropped, and it is nowhere in the archive. Do you know where to find it ?

                              • On May 23rd, 2009, linuxcentre said:

                                Hulu obfuscated beyond belief well before I dropped support for it so no versions of get_iplayer will work today unless you happen to get a supported hulu plugin from elsewhere.

                              • On May 29th, 2009, Bill said:

                                linuxcentre, in your post you state it’s possible to hack at hulu with enough determination; can you share some of the details on that?

                                The point I’m most interested is decoding the SMIL file to get to the actual streaming video urls on akamai.

                                Also, I would like to know if it’s possible to use hulu’s servers from outside the USA (after defeating the http geocheck).

                                Thank you very much and cheers,
                                Bill

                                • On May 29th, 2009, linuxcentre said:

                                  Hi Bill, I’ve dropped support for hulu. It’s best to have a google around for the latest methods of deobfuscation. AFAIK, you cannot defaet the geoIP check with hulu.

                                • On August 15th, 2009, Jayme said:

                                  Thanks for this great program. Where can I find plugins for this? Where can I find any plugins for get-iplayer?? Thanks in advance!

                                  • On August 16th, 2009, linuxcentre said:

                                    The two plugins that I support come with get_iplayer automatically – itv and podcast.

                                  • On November 15th, 2009, krivine said:

                                    I also get the “ERROR: Invalid type ‘itv’ specified.” problem. I downloaded and installed the latest package for Debian (Ubuntu Jaunty) today, and attempts to update confirm it’s the latest one. Any ideas?

                                    • On November 15th, 2009, linuxcentre said:

                                      See the FAQ

                                    WordPress Themes