Feb 06 2009

Lots of new features have been added to get_iplayer since the New Year. Notably Hulu TV and Film indexing and download support have been added for US based users. This can add up to 9000 additional TV programmes and 200+ films!! Although I have now limited it to the most popular 2000 TV programmes to save on indexing speed.

To list all the programmes available on Hulu you can simply type:

get_iplayer --type=hulu

to get the index. Unfortunately, Hulu do not provide a very extensive XML feed so the indexing can take a few minutes. You can download a programme using the usual syntax, or, if you don’t wish to index the Hulu.com site, you can specify a pid or URL directly. e.g.:

get_iplayer --type=hulu --pid http://www.hulu.com/watch/20727

You will need rtmpdump (see the get_iplayer download page) to be able to download any Hulu programmes.

As usual, this feature is rather new and will have bugs so please let me know if you find any.

Just one other note: If you wish to see all of the options available on get_iplayer you now need to use the –help-long option.

17 Comments

  • On February 13th, 2009, MtnMan said:

    Hi Phil (or forum),

    Congrats on your awesome program and I hope it really gets the exposure it deserves. Can you help me out with one thing…maybe you can shed some light? I am in the US and can watch Hulu.com videos perfectly, by the way.

    I am using Ubuntu 8.10 with get_iplayer 1.34 and rtmpdump 1.33 (installed correctly and working) and when i try to grab a Heroes episdoe from hulu.com using the following CLI of

    get_iplayer –type=hulu –pid http://www.hulu.com/watch/56972/heroes-trust-and-blood

    the program stops/hangs and states

    get_iplayer v1.34, Copyright (C) 2009 Phil Lewis
    This program comes with ABSOLUTELY NO WARRANTY; for details use –warranty.
    This is free software, and you are welcome to redistribute it under certain
    conditions; use –conditions for details.

    WARNING: Cannot read /home/sysadmin/.get_iplayer/download_history

    INFO Trying to download pid using type hulu
    INFO: pid not found in hulu cache
    WARNING: Cannot read /home/sysadmin/.get_iplayer/download_history

    INFO: flashnormal,flashlow modes will be tried
    INFO: Trying flashnormal mode to download hulu: –
    INFO: File name prefix = Heroes_-_Trust_and_Blood_hulu-56972_flashnormal

    INFO: Command: rtmpdump –resume –rtmp “rtmp://cp39465.hulu.com:1935/ondemand?_fcs_vhost=cp39465.hulu.com&auth=daEaDdwdMbac4cbdsdGanaTa_c.dSdNbJcv-bjLq73-8-5qHEzDn0Gzn&aifp=NS20070910&slist=content/39484/14/442/HuluTranscode_67105_77122_FLASH_700K_16x9_23_976__-uO-cpMiJUGBId7n-TPRiw” –auth “” –swfUrl “http://www.hulu.com/player.swf” –tcUrl “rtmp://204.141.87.61:1935/ondemand?_fcs_vhost=cp39465.hulu.com&auth=daEaDdwdMbac4cbdsdGanaTa_c.dSdNbJcv-bjLq73-8-5qHEzDn0Gzn&aifp=NS20070910&slist=content/39484/14/442/HuluTranscode_67105_77122_FLASH_700K_16x9_23_976__-uO-cpMiJUGBId7n-TPRiw;.international=false” –app “ondemand?_fcs_vhost=cp39465.hulu.com&auth=daEaDdwdMbac4cbdsdGanaTa_c.dSdNbJcv-bjLq73-8-5qHEzDn0Gzn&aifp=NS20070910&slist=content/39484/14/442/HuluTranscode_67105_77122_FLASH_700K_16x9_23_976__-uO-cpMiJUGBId7n-TPRiw;.international=false” -o “./Heroes_-_Trust_and_Blood_hulu-56972_flashnormal.partial.avi.flv” 1>&2
    RTMP Dumper v1.3
    (c) 2009 Andrej Stepanchuk, license: GPL

    DEBUG: Setting buffer time to: 36000000ms
    Connecting to rtmp://cp39465.hulu.com:1935/ondemand?_fcs_vhost=cp39465.hulu.com&auth=daEaDdwdMbac4cbdsdGanaTa_c.dSdNbJcv-bjLq73-8-5qHEzDn0Gzn&aifp=NS20070910&slist=content/39484/14/442/HuluTranscode_67105_77122_FLASH_700K_16x9_23_976__-uO-cpMiJUGBId7n-TPRiw …
    DEBUG: Hostname: cp39465.hulu.com
    DEBUG: Port: 1935

    • On February 13th, 2009, linuxcentre said:

      That all looks OK except for the hanging after ‘DEBUG: Port: 1935’, sometimes rtmpdump can appear to take ages to connect while it is actually downloading content. Otherwise there may be a connectivity issue getting to port 1935 (firewall?). Remember that rtmp cannot go through a normal web proxy.

    • On February 17th, 2009, Alan said:

      Hello,

      The script does not seem to be indexing all the hulu content. I’m doing a search for Babylon 5 and it comes back with 1 episode. If i go to the hulu.com website, I can that there are 45 episodes.

      • On February 17th, 2009, linuxcentre said:

        It only indexes the first 200 episodes in each hulu ‘channel’ (i.e. category). Try flushing the cache as follows to get the full 9000+ programmes: “get_iplayer –type hulu –flush –hulu-pages 200” just go make a cup of tea while you wait… There are more hulu specific options if you run get_iplayer –long-help

      • On February 17th, 2009, Alan said:

        Great, thanks.

        Next question 🙂

        Does get_iplayer support rtmpe streams? Seems the flashhigh versions are requiring rtmpe or rtmpte support.

        • On February 17th, 2009, linuxcentre said:

          Sadly rtmpdump does not (get_iplayer can only resolve the URLs) – rtmpe is yet another proprietary Adobe rtmp transport – this time encrypted.

        • On February 19th, 2009, Dan said:

          Does it support QMX streams?

          • On February 19th, 2009, Aaron said:

            I’m having trouble getting get_iplayer to work with Hulu. It builds the index and can search shows, but the actual downloading fails. I’m within the US, so there are no proxy issues. I also tried with –vmode=flashhigh and –vmode=flashlow to no avail. There is a tilda in the retrieved HPID–is that normal? Am I doing something wrong?

            ./get_iplayer –type=hulu –vmode=flashnormal –rtmpdump=./rtmpdump -g 201008 -v

            get_iplayer v1.39, Copyright (C) 2009 Phil Lewis
            This program comes with ABSOLUTELY NO WARRANTY; for details use –warranty.
            This is free software, and you are welcome to redistribute it under certain
            conditions; use –conditions for details.

            Current options:
            rtmpdump = ./rtmpdump
            type = hulu
            verbose = 1
            vmode = flashnormal

            INFO: Search args: ‘201008’
            INFO: Additionally getting cached programme data for hulu
            INFO: got 2352 cache entries for hulu
            Matches:
            201008: House – Painless, Drama

            INFO: 1 Matching Programmes
            WARNING: Cannot read ~/.get_iplayer/download_history

            INFO: flashnormal modes will be tried
            INFO: Trying flashnormal mode to download hulu: House – Painless
            INFO: Getting page http://www.hulu.com/watch/54717
            .INFO: CID=13740183
            INFO: Getting page http://r.hulu.com/videos?content_id=13740183
            .INFO: HPID=98fcd87749f12ed01bbbded49cac597bff8a9d33489067f928c91f43990d6a
            21~998dddcb22e2f436f6ef02c0436d14079d37fb3bcef51f96ce1d6202e10cb996
            INFO: Getting page http://releasegeo.hulu.com/content.select?pid=98
            fcd87749f12ed01bbbded49cac597bff8a9d33489067f928c91f43990d6a21~998d
            ddcb22e2f436f6ef02c0436d14079d37fb3bcef51f96ce1d6202e10cb996&mbr=tr
            ue&format=smil
            WARNING: Failed to get programme stream data for House from site
            WARNING: Failed to get programme stream data for House from site
            WARNING: Failed to get programme stream data for House from site
            WARNING: No programme stream URLs were found for flashnormal, skipping
            DEBUG: Download using flashnormal mode return code: ‘next’
            INFO: skipping flashnormal mode
            ERROR: Failed to download ‘House – Painless (hulu-54717)’

            • On February 19th, 2009, linuxcentre said:

              Yes, Hulu isn’t supported until the new Hulu contentid->PID algorithm is reverse engineered. Hulu kicked off Boxee today and none of our scripts now work! Oh well – guess you’ll have to use other methods to get your US TV fix…

              • On February 19th, 2009, Aaron said:

                Well that’s a shame–it breaks on the first day I try it! Is there a way to collect examples of valid contentid and PID pairs in order to attempt reverse engineering?

                • On February 20th, 2009, Howard Chu said:

                  If you’re familiar with tcpdump / wireshark you can sniff the traffic from a regular Hulu session and try to work it out. I’ve got one trace on my system that I’m playing with at the moment. Hulu is unwatchable in my web browser because the Flash plugin is such a pig. The videos only playback smoothly if I download them and use a regular movie player…

                  • On February 20th, 2009, linuxcentre said:

                    It’s all in the flash. I think we’ll have it working soon!

            • On February 28th, 2009, hulugeek said:

              instead of wireshare you can use tamper data addon in firefox… much much better and you can also tamper the outgoing http data aswell.

              • On April 13th, 2009, georges said:

                plz

                • On August 27th, 2009, Angelo said:

                  $ get_iplayer.pl –type=hulu
                  get_iplayer v2.20, Copyright (C) 2009 Phil Lewis
                  This program comes with ABSOLUTELY NO WARRANTY; for details use –warranty.
                  This is free software, and you are welcome to redistribute it under certain
                  conditions; use –conditions for details.

                  ERROR: Invalid type ‘hulu’ specified. Valid types are: livetv,itv,tv,liveradio,podcast,radio

                  • On September 19th, 2009, Jake said:

                    I’m getting the exact same error as Angelo. Anyone help?

                    • On September 19th, 2009, Matt said:

                      Hulu is not supported by get_iplayer. It was tried for a short while along with support for channel 4’s new catchup service but support was dropped due to problems.

                  WordPress Themes