Jan 06 2009

get_iplayer Indexes, Searches and Downloads ITV player TV Programmes

Happy New Year!
For all of those who desperately want to watch UK ITV programmes, get_iplayer (v1.13 onwards) can now download them for you! (just specify –type=itv)

Normally ITV programmes can be streamed in real-time, on-demand, from the itv.com website. However you need microsoft Silverlight to watch them in a web browser. On the back-end itv.com provides these programmes as obfuscated asf (wmv/wma) streams.

get_iplayer scrapes the Catch-Up parts of the ITV site and indexes all of the available programmes. You can then search and download them as you would with BBC iPlayer programmes.

For example,

get_iplayer --type=itv 'Rising Damp' --get

Due to the advertisement breaks, many of the ITV programmes are split into a handful of approximately 15 minute segments. get_iplayer will find all of the segments and download them in parallel. At present I have not found a way to download the segments in anything except real-time so the parallelism means that most programmes only take around 15 minutes to download.

And before you ask – No, I’m not planning on adding 4oD or DemandFive because they use Windows based DRM and I’m not going to play that game. DRM will eat itself.

Update:

get_iplayer v1.17+ now indexes the ITV ‘TV Classics’ programmes in addition to the ITV ‘Catch Up’ programmes. This adds over 300 episodes to the avaiable list.

Update (16th September 2009):

Looks like ITV have completely redesigned their site and back end which has broken the existing ITV plugin. Maybe someone would like to write a new plugin?

Tags: catch-up, download, get_iplayer, iplayer, itv, itv player, itvplayer, tv, tv classics

Filed at 8:36pm in General, iplayer | linuxcentre

26 Comments

On January 6th, 2009, Bigarules said:

Woohoo!

[Reply]
On January 7th, 2009, appreciated said:

FYI for anyone else sitting behind a proxy – mplayer doesn’t use/respect proxy authentication settings.

[Reply]
- On January 7th, 2009, linuxcentre said:
  
  I’m, not behind a proxy but mplayer does not use web proxies for mms or rtsp streaming. get_iplayer uses these for ITV and BBC Radio/Realaudio. Same goes for the rtmpdump tool.
  
  [Reply]
On January 7th, 2009, Raj said:

can you add a parameter that will select the number of episodes to download for the search string:

I would prefer to to limit download to the latest episode only. This is because I use get_iplayer in a USB stick and not refresh it everyday.

./get_iplayer_current newsnight
Matches:
353: Newsnight Scotland – 05/01/2009, ‘BBC Two’, News,TV, default
354: Newsnight Scotland – 06/01/2009, ‘BBC Two’, News,TV, default
355: Newsnight – 05/01/2009, ‘BBC Two’, News,TV, default
356: Newsnight – 06/01/2009, ‘BBC Two’, News,TV, default
From above example, I want only 356 to be downloaded.

Using –since is unfortunately not a complete solution as different programs have difffertnt numbers of episodes per week….

Many thanks for your great work

[Reply]
- On January 8th, 2009, linuxcentre said:
  
  Hmm, I had a go at this but it didn’t work. The problem is that the BBC do not advertise an airing timestamp in the metadata. Even the –since option can be inaccurate because sometimes episodes are made available in the wrong order on iplayer feeds. So I guess the answer is no until the beeb improve their metadata or feeds.
  
  [Reply]
On January 8th, 2009, Raj said:

hmmm… I did not completely look into the code but this is what I thought of doing.

INFO: 10 Matching Programmes
From the line “Matching Programmes” get the number of programes.

Next, if it is greater than 0, then, list them and sort then in descending order and print only the highest number.

Following is a bash wrapper I use for the moment. Could you have a go of the same in perl. I am a bit busy with my exams.
——–
grep and get the prog .ids. (353, 354 above example)

for prog_name in “newsnight” “Top Gear” “Mock the week”
do
#echo $prog_name
#for every program check if it has more than one match
number_of_programmes=$(./get_iplayer_current $prog_name | grep Matching | grep INFO| awk ‘{print $2 }’)
#echo $number_of_programmes $prog_name

# if yes, then from cache sort the avaliable programs and download
if [ $number_of_programmes -gt 0 ]
then
# grep -i “$prog_name” .get_iplayer/tv.cache
prog_id=$(grep -i “$prog_name” .get_iplayer/tv.cache | awk -F”|” ‘{print $1}’ | sort -nr | head -1)
echo “tv ” $prog_id $prog_name $number_of_programmes
./get_iplayer_current –get $prog_id -o $datadir

fi
done

[Reply]
- On January 8th, 2009, linuxcentre said:
  
  That’s OK as long as the programme mentions the episode number. Some progs use the episode name and some use a date (sometimes in a non standard way). SO basically nothing will work reliably unfortunately. The ordering user by the index number is basd on alphanumeric sorting which works some of the time.
  
  [Reply]
- On January 8th, 2009, linuxcentre said:
  
  Having sdaid all that, do you keep the ~/.get_iplayer/download_history file between downloads? If you do then you’ll hardly ever end up downloading an unnecessary episode.
  
  [Reply]
On January 11th, 2009, Chris said:

Could something like following be used to concatenate the itv wmv/wma streams into a single container?

mencoder -oac copy -ovc copy -idx -o foo.avi bar*.asf

or the ffmpeg / vlc equivalent depending on what the latest script dependencies are

[Reply]
- On January 11th, 2009, linuxcentre said:
  
  I did want to do this but it seems that some programmes cause mencoder to go 100% cpu and hang when indexing. Also, on the downside, this type of conversion adds around 37% to the size of the data.
  
  [Reply]
On January 20th, 2009, Darron Pierce said:

I need to watch the news with this iplayer

[Reply]
- On January 20th, 2009, linuxcentre said:
  
  Try:
  get_iplayer –type=tv news
  Then:
  get_iplayer –type=tv –get
  
  [Reply]
On January 29th, 2009, rioting_pacifist said:

Get itv doesn’t work for me it gets the first 8mb ish then stops.

btw a nice feature would be for aliases to work for channels and categories
e.g BBC1 -> bbc one, BBC4 -> BBC Four, radio1 -> BBC Radio 1, etc
& childrens/kids -> Children’s and whatever itv call kids tv

p.s keep up the great work

[Reply]
On April 2nd, 2009, ufufgnaf said:

it doesn’t work sometimes.

INFO: 1 Matching Programmes
INFO: mms modes will be tried
INFO: Trying mms mode to download itv: The Boat Race 2009 – 38709
WARNING: No programme stream URLs were found, skipping
INFO: skipping this programme

what shall i do?

[Reply]
On May 23rd, 2009, Angela said:

I’m having loads of problems downloading itv programmes. I’m trying to get the Jewel in the Crown series but the threaded attempts fail. They continually get to about 75Mb and restart from zero. I’ve tried ‘–itvnothread’ but the single download fails too. Is there a way to get the download to resume from where it got to rather than start over?

But I have feeling 75Mb would be about right for one programme segment of tis quality. Is it getting to the end but not realising it?

[Reply]
- On May 23rd, 2009, linuxcentre said:
  
  I’m trying this now – any particular episode?
  Since you use Linux you should not have to ever use –itvnothread – thats really intended for Windows users who are stuck with broken ActivPerl or Strawberry perl (it is OK on cygwin though).
  If I cannot reproduce the problem I’ll maybe have to get you to run it again with the –verbose flag on and then email me the output.
  
  [Reply]
On June 9th, 2009, Andrew said:

I just got an email to say 4od is now available for Mac & Linux via a flash based interface.

Is there any chance you will be supporting 4od now?

Thanks for the great work.

[Reply]
- On June 9th, 2009, linuxcentre said:
  
  That email was a bit late. It was working on Linux and MAC months ago. Since then I supported 4oD and have since dropped it… See my blog posts on this.
  
  [Reply]
On August 2nd, 2009, schnubbel said:

Hmm – currently this doesn’t work for me .. indexing works, selecting a particular stream does. But – as a post above stated – it then just skips the selected show:

WARNING: No programme stream URLs were found, skipping
INFO: skipping this programme

BBC iplayer works, though! Big thanks for that ..

[Reply]
- On August 2nd, 2009, linuxcentre said:
  
  Could you let me know your OS, command line used and which programme name/episide (not the index number) you were trying to get?
  
  [Reply]
  - On August 3rd, 2009, schnubbel said:
    
    Oops, just realized that I accidentally replied in the wrong thread 😉 As I don’t like to double-post, see below ..
    
    [Reply]
On August 2nd, 2009, schnubbel said:

Hi 🙂

O/S is gentoo linux, get_iplayer 2.13 (and 2.14). Command line was:

get_iplayer –type=itv ‘midsomer murders’ –get

Also I’m using a proxy specified with “-p http://…” – same combination works for bbc iplayer.

Cheers and best regards ..

[Reply]
- On August 3rd, 2009, linuxcentre said:
  
  The ITV streams use MMS protocol (not HTTP), http proxies don’t work for MMS afaik. Even if they did, get_iplayer doesn’t support it.
  
  [Reply]
On September 16th, 2009, John said:

Hi Phil,
I am having problems downloading ITV. I get “No programme stream URLs were found, skipping”

Command line:
get_iplayer –verbose –showopts –get –type=itv “Corona”

Log excerpt:
INFO: User prefs dir: /home/john/.get_iplayer
INFO: System options dir: /var/lib/get_iplayer/options
Current options:
exclude = Omnibus
output = /var/cache/video/iplayer
packagemanager = apt-get
showoptions = 1
since = 24
type = itv
verbose = 1

INFO: Search args: ‘Corona’
Options from Files:
exclude = Omnibus
output = /var/cache/video/iplayer
packagemanager = apt-get
since = 24

Options from Command Line:
showoptions = 1
type = itv
verbose = 1

Options Used:
exclude = Omnibus
output = /var/cache/video/iplayer
packagemanager = apt-get
showoptions = 1
since = 24
type = itv
verbose = 1

Search Args: Corona

INFO: got 278 cache entries for itv
INFO: Search term ‘Corona’ is a substring
DEBUG: Search download_regex = Corona
DEBUG: Search channel_regex = .*
DEBUG: Search category_regex = .*
DEBUG: Search versions_regex = .*
DEBUG: Search exclude_regex = Omnibus
DEBUG: Search channel_exclude_regex = ^ROGUE$
DEBUG: Search category_exclude_regex = ^ROGUE$
Matches:
100050: Coronation Street – Thu 03 Sep 2009, ITV Catch-up, TV,
100051: Coronation Street – Mon 31 Aug 2009, ITV Catch-up, TV,
100052: Coronation Street – Mon 17 Aug 2009, ITV Catch-up, TV,
100053: Coronation Street – Mon 17 Aug 2009, ITV Catch-up, TV,
100054: Coronation Street – Fri 21 Aug 2009, ITV Catch-up, TV,
100055: Coronation Street – Thu 27 Aug 2009, ITV Catch-up, TV,
100056: Coronation Street – Fri 21 Aug 2009, ITV Catch-up, TV,
100057: Coronation Street – Mon 24 Aug 2009, ITV Catch-up, TV,
100058: Coronation Street – Mon 24 Aug 2009, ITV Catch-up, TV,
100059: Coronation Street – Fri 04 Sep 2009, ITV Catch-up, TV,
100060: Coronation Street – Fri 04 Sep 2009, ITV Catch-up, TV,
100061: Coronation Street – Mon 07 Sep 2009, ITV Catch-up, TV,
100062: Coronation Street – Thu 10 Sep 2009, ITV Catch-up, TV,
100063: Coronation Street – Mon 07 Sep 2009, ITV Catch-up, TV,
100064: Coronation Street – Fri 11 Sep 2009, ITV Catch-up, TV,
100065: Coronation Street – Fri 11 Sep 2009, ITV Catch-up, TV,
100066: Coronation Street – Mon 14 Sep 2009, ITV Catch-up, TV,
100067: Coronation Street – Mon 14 Sep 2009, ITV Catch-up, TV,

INFO: 18 Matching Programmes
INFO: Programme not in download history
INFO: Will search for versions: default
INFO: Mode list: itv
INFO: Checking existence of default version
INFO: Version = default
INFO: ITV Video Stage 1 URL: http://www.itv.com/_app/video/GetMediaItem.ashx?vodcrid=crid://itv.com/100039&bitrate=384&adparams=SITE=ITV/AREA=CATCHUP.VIDEO/SEG=CATCHUP.VIDEO HTTP/1.1
INFO: Getting page http://www.itv.com/_app/video/GetMediaItem.ashx?vodcrid=crid://itv.com/100039&bitrate=384&adparams=SITE=ITV/AREA=CATCHUP.VIDEO/SEG=CATCHUP.VIDEO HTTP/1.1
INFO: itv modes will be tried for version default
INFO: Trying itv mode to record itv: Coronation Street – Thu 03 Sep 2009
INFO: Attempt number: 1 / 3
INFO: ITV Video Stage 1 URL: http://www.itv.com/_app/video/GetMediaItem.ashx?vodcrid=crid://itv.com/100039&bitrate=384&adparams=SITE=ITV/AREA=CATCHUP.VIDEO/SEG=CATCHUP.VIDEO HTTP/1.1
INFO: Getting page http://www.itv.com/_app/video/GetMediaItem.ashx?vodcrid=crid://itv.com/100039&bitrate=384&adparams=SITE=ITV/AREA=CATCHUP.VIDEO/SEG=CATCHUP.VIDEO HTTP/1.1
WARNING: No programme stream URLs were found, skipping
DEBUG: Record using itv mode return code: ‘skip’
INFO: skipping this programme
ERROR: Failed to record ‘Coronation Street – Thu 03 Sep 2009 (100039)’
INFO: Programme not in download history

(repeats many times)

[Reply]
- On September 16th, 2009, linuxcentre said:
  
  Looks like ITV have completely redesigned their site and back end. Maybe someone would like to write a new plugin?
  
  [Reply]
On November 29th, 2009, liam said:

Are you making a new plugin ??? plz do!!

[Reply]

Linuxcentre

get_iplayer Indexes, Searches and Downloads ITV player TV Programmes

Update:

Update (16th September 2009):

26 Comments

Pages

Recent Posts

Categories

Charity