|
Apr
29
2009
|
Update 6th May 2009: You might want to read this first…
A milestone has been reached today with the addition of Channel 5’s Demand Five service to the list of DRM-free services that get_iplayer can index, search and record. The following are now supported:
- BBC iPlayer TV – up to 3200 kbps HD 720p H.264 / Quicktime / MP4
- ITV player Catch-up / Classic TV – 384 kbps+ WMV / ASF
- Channel4 4oD Catch-up TV – up to 500 kbps H.264 / MP4
- Channel5 Demand Five TV – up to 720 kbps H.264 / MP4
- BBC iPlayer Radio – 128 – 192 kbps MP3 / AAC and Realaudio
- BBC Podcasts – 96 – 192 kbps MP3 / AAC
- BBC iPlayer TV subtitles – SubRip / srt
Supported platforms are: Linux, MacOSX, Windows and pretty much any modern OS that can run perl. See the download and installation pages.
To download and search programmes use:
get_iplayer –type=TYPE <other options>
Where TYPE is either ‘tv’ (BBC TV), ‘itv’, ‘ch4’, ‘five’, ‘radio’ (BBC Radio), or ‘podcast’ (BBC).
For full instructions see the documentation page
Now I’m off for a beer….
Your rtmpdump is too old – needs to be v.1.5 or better.
If you are on MacOSX then you need to compile rtmpdump v1.5 from source. I don’t have access to a mac so if anyone can help here please email the binary to iplayer2 (at sign) linuxcentre.net
Am on a Mac. Working on compiling rtmpdump from source, however it throws up some errors which I believe are related to needing to update OpenSSL. Linux is just so much easier sometimes…
I’ll email you the file if I can get it compiled
The README file in the tarball for rtmpdump has some pointers generally on compiling. Do you by any chance need the development package for openssl (i.e. the headers etc) before you can compile against this lib?
The headers etc. are all in place. I’m pretty sure the issue is that the version of OpenSSL included with OS X (10.5.6 in my case) is compiled without SHA256 support.
The only nice way around this I can think of is to statically link a OpenSSL library (compiled with SHA256 support) to the rtmpdump binary, so this can be distributed without the need for messing around with the system files – something which really shouldn’t be necessary.
I’ll have a go – but if you have any pointers on how to build it like this that’d be great 🙂
OK, managed to build it, using above method 🙂 Am emailing you the binary to put on the downloads page
You’re a star….!
Using the latest version of rtmpdump and get_ipalyer but having issues with downloads from five. They’re casuing rtmpdump to crash every time – BBC, ITV and CH4 all working ok though.
Have you tried reinstalling the whole Windows Automated Installer – it was overhauled a few days ago?
yeah I initially tried it just using the –update and manually downloading rtmpdump 1.5 but when it wouldn’t work used the Automated Installer with the same results.
I am currently in touch with the author of rtmpdump trying to fix this crash. Can let you know once it is fixed. I have a version of rtmpdump that does not crash on my computer anymore and downloads from channel five, but I do not know enough sockets programming to be 100% sure that I fixed it correctly. Can send you my version if you would like to try it.
rtmpdump ver.1.5a is now available from the sourceforge, which should fix the channel 5 problem.
That’s great – just tried the new version and it seems to work now (you have to get the Win32 binary from rtmpdump’s Sourceforge repository though – the link in the Downloads section here is still the older one).
Good job – this is a great application. I hope some clever person finds a way to compile it for the Popcorn Hour media player – getting iPlayer working on it would be great.
zinger5
I’ve posted Windows and Linux/Fedora rtmpdump v1.5a on the download page now.
Hi Zinger, I know this is a very long time ago, i have spent a considerable amount of time playing with the flvstreamer and get_iplayer for the QNAP nas and also now the PopCorn Hour. I have successfully got get_iplayer working on the MIPes processor, i still have a little way to go, but it’s pretty much there. I have 1 last issue with regards FFmpeg and then i will try and put some documentation together if you are interested.
Tim.
The fix in 1.74 for Channel 4 doesn’t seem to be working very well – it looks like it picks a random (non-integer) number between 0 and 6, then looks in %user_agent for that number as a key, so it always gets undefined, and ends up sending no User-Agent header at all, and gets blocked by Ch4.
(Apart from that, this is an excellent tool 🙂 )
Oh, also the Referer is set to “$baseurl.’/programmes/catch-up” instead of “$baseurl/programmes/catch-up”. If I fix that and the UA, then I can successfully download the programme listings, though then it fails to download the actual programme.
Thanks, user agent selection bugfixed in v1.75.
Still a little bug: the code now says $user_agent_list[ int(rand($#user_agent_list))], so if (for example) @user_agent_list had two entries then $#user_agent_list == 1 and int(rand(1)) is always 0. So it’ll never pick the last entry in the list.
(Easily fixed with int(rand($#user_agent_list+1)) or int(rand(@user_agent_list)) instead. And actually you can just say $user_agent_list[rand @user_agent_list] because array indexes automatically get rounded down. But, uh, I can’t work out where that feature is documented. Surely it must be somewhere…)
Ah – yes – sorry – too much beer and tiredness in the last few days. I’ve fixed and tested that now in v1.77. The trick is to add 0.5.
Hmm, that doesn’t seem like a great trick. If you have three items in the list and use int(rand(2+0.5)), there’s a 40% chance it’ll give 0, 40% it’ll give 1, and 20% 2, so it’s not very uniform. If you use int(rand(2)+0.5) instead, then it’s 25% 0, 50% 1, and 25% 2, so it’s still not very uniform. I think you really want to use int(rand(3)) instead 🙂 (and rand is defined such that it might give 2.999… but never precisely 3 so it’ll always work fine)
Used a better way now. Now its nice and uniform 🙂 Thanks.
$uas = $user_agent_list[rand @user_agent_list];
The first time I have tried to download from the FIVE channel:
INFO: Command: .\rtmpdump.exe -r “rtmpe://streamflv.download.five.tv/a800/o10/mp
4:bthttp/five/downloadstore/secure/peppapig/movies/Peppa-Pig_S2_E1_Episode-1_FLV
STREAM.mp4?e=1241195948&h=a962d8e0c885052f0aabcb9843d0e799” –resume –swfsize 8
48259 –swfhash 75a4a2a2dc7e7c198478161bb1c8826721034f4f36bed7d23c6bf42910a500a1
–protocol “3” -o “Downloads\/Peppa_Pig_Episode_1_C5115230001.partial.mp4.flv”
1>&2
RTMPDump v1.5a
(c) 2009 Andrej Stepanchuk, license: GPL
DEBUG: Parsing…
DEBUG: Parsed protocol: 3
DEBUG: Parsed host : streamflv.download.five.tv
DEBUG: Parsed app : a800/o10
DEBUG: Parsed playpath: mp4:bthttp/five/downloadstore/secure/peppapig/movies/Pep
pa-Pig_S2_E1_Episode-1_FLVSTREAM.mp4?e=1241195948&h=a962d8e0c885052f0aabcb9843d0
e799
DEBUG: Setting buffer time to: 36000000ms
Connecting …
DEBUG: Protocol : RTMPE
DEBUG: Hostname : streamflv.download.five.tv
DEBUG: Port : 1935
DEBUG: Playpath : mp4:bthttp/five/downloadstore/secure/peppapig/movies/Peppa-Pig
_S2_E1_Episode-1_FLVSTREAM.mp4?e=1241195948&h=a962d8e0c885052f0aabcb9843d0e799
DEBUG: tcUrl : rtmpe://streamflv.download.five.tv:1935/a800/o10
DEBUG: app : a800/o10
DEBUG: flashVer : LNX 9,0,124,0
DEBUG: live : no
DEBUG: timeout : 300 sec
DEBUG: SWFSHA256:
75 A4 A2 A2 DC 7E 7C 19 84 78 16 1B B1 C8 82 67 21 03 4F 4F 36 BE D7 D2 3C 6B F4
29 10 A5 00 A1
DEBUG: SWFSize : 848259
ERROR: Connect, failed to connect socket. Error: 10065
Failed to connect!
WARNING: Failed to download file Downloads\/Peppa_Pig_Episode_1_C5115230001.part
ial.mp4.flv via RTMP
INFO: skipping flashhigh mode
ERROR: Failed to download ‘Peppa Pig – Episode 1 (C5115230001)’
Has anyone else had this error or is it just me? Does this socket error mean than the server is unavailable? Any suggestions gratefully received.
Is there any hope of the 64bit Linux rtmpdump being posted up? The 32bit version doesn’t work for me and for some reason I can’t get the 64bit version to compile from source.
This is not good!
Can anyone help?
Iam trying to download peppa pig from five but i keep getting this message?
I have the latest iplayer installer and have downloaded 1.5a of rtmp from scourceforge, am i doing something wrong?
I copied and pasted latest rtmpdump in get_iplayer directory, is this the correctthing to do?
cheers
C:\Program Files\get_iplayer>get_iplayer –type=five –get 50444
C:\Program Files\get_iplayer>perl.exe get_iplayer.pl –type=five –get 50444
get_iplayer v1.75, Copyright (C) 2009 Phil Lewis
This program comes with ABSOLUTELY NO WARRANTY; for details use –warranty.
This is free software, and you are welcome to redistribute it under certain
conditions; use –conditions for details.
Matches:
50444: Peppa Pig – Episode 13, five, tv,
INFO: 1 Matching Programmes
INFO: Checking existence of default version
INFO: flashhigh modes will be tried for version default
INFO: Trying flashhigh mode to download five: Peppa Pig – Episode 13
WARNING: Required program .\rtmpdump.exe does not exist (see http://linuxcentre.
net/getiplayer/installation and //linuxcentre.net/getiplayer/download)
INFO: skipping flashhigh mode
ERROR: Failed to download ‘Peppa Pig – Episode 13 (C5115230013)’
C:\Program Files\get_iplayer>
C:\Program Files\get_iplayer>get_iplayer –type=five –get 50444 –raw
C:\Program Files\get_iplayer>perl.exe get_iplayer.pl –type=five –get 50444 –r
aw
get_iplayer v1.75, Copyright (C) 2009 Phil Lewis
This program comes with ABSOLUTELY NO WARRANTY; for details use –warranty.
This is free software, and you are welcome to redistribute it under certain
conditions; use –conditions for details.
Matches:
50444: Peppa Pig – Episode 13, five, tv,
INFO: 1 Matching Programmes
INFO: Checking existence of default version
INFO: flashhigh modes will be tried for version default
INFO: Trying flashhigh mode to download five: Peppa Pig – Episode 13
WARNING: Required program .\rtmpdump.exe does not exist (see http://linuxcentre.
net/getiplayer/installation and //linuxcentre.net/getiplayer/download)
INFO: skipping flashhigh mode
ERROR: Failed to download ‘Peppa Pig – Episode 13 (C5115230013)’
C:\Program Files\get_iplayer>
Just upgraded to 1.75, can get itv, five and all bbc stuff, but ch4 just gives me:
get_iplayer v1.75, Copyright (C) 2009 Phil Lewis
This program comes with ABSOLUTELY NO WARRANTY; for details use –warranty.
This is free software, and you are welcome to redistribute it under certain
conditions; use –conditions for details.
INFO: Getting ch4 Programmes List
WARNING: Failed to get series list from from channel4 site
WARNING: Failed to get series list from from channel4 site
WARNING: Failed to get series list from from channel4 site
INFO: 0 Matching Programmes
Any ideas why???!!!
Fix to 1.75 issue above.
— get_iplayer-1.75.orig Sat May 2 16:25:52 2009
+++ get_iplayer-1.75 Sat May 2 16:26:25 2009
@@ -1267,7 +1267,11 @@
my $ua = LWP::UserAgent->new;
$ua->timeout([$lwp_request_timeout]);
$ua->proxy( [‘http’] => $opt->{proxy} ) if $opt->{proxy};
– $ua->agent( $user_agent{$agent} );
+ if (length($user_agent{$agent}) == 0) {
+ $ua->agent( $agent );
+ } else {
+ $ua->agent( $user_agent{$agent} );
+ }
$ua->conn_cache(LWP::ConnCache->new());
#$ua->conn_cache->total_capacity(50);
$ua->cookie_jar( HTTP::Cookies->new( file => $cookiejar, autosave => 1, ignore_discard => 1 ) ) if not $nocookiejar;
Unmangle diff
— get_iplayer-1.75.orig Sat May 2 16:36:24 2009
+++ get_iplayer-1.75 Sat May 2 16:33:34 2009
@@ -1268,5 +1268,9 @@
$ua->timeout([$lwp_request_timeout]);
$ua->proxy( [‘http’] => $opt->{proxy} ) if $opt->{proxy};
– $ua->agent( $user_agent{$agent} );
+ if (length($user_agent{$agent}) == 0) {
+ $ua->agent( $agent );
+ } else {
+ $ua->agent( $user_agent{$agent} );
+ }
$ua->conn_cache(LWP::ConnCache->new());
#$ua->conn_cache->total_capacity(50);
Hm still mangled, but you get the idea
thanks – I’ve hopefully fixed that prblem now.
Five downloads are not working for me on Ubuntu 9.04, rtmpdump (v1.5a) just seems to hang afetr SWFSIZE:
./get_iplayer –type=five –rtmpdump=’./rtmpdump’ –get 50187
get_iplayer v1.75, Copyright (C) 2009 Phil Lewis
This program comes with ABSOLUTELY NO WARRANTY; for details use –warranty.
This is free software, and you are welcome to redistribute it under certain
conditions; use –conditions for details.
Matches:
50187: Fifth Gear – Episode 1, five, tv,
INFO: 1 Matching Programmes
INFO: Checking existence of default version
INFO: flashhigh modes will be tried for version default
INFO: Trying flashhigh mode to download five: Fifth Gear – Episode 1
INFO: File name prefix = Fifth_Gear_Episode_1_C5143730001
INFO: Command: ./rtmpdump -r “rtmpe://streamflv.download.five.tv/a800/o10/mp4:bthttp/five/downloadstore/secure/fifthgear/movies/Fifth-Gear_S15_E1_Episode-1_FLVSTREAM.mp4?e=1241259330&h=709052a6c91019d9b7adb43eb1248a54” –resume –swfsize 848259 –swfhash 75a4a2a2dc7e7c198478161bb1c8826721034f4f36bed7d23c6bf42910a500a1 –protocol “3” -o “./Fifth_Gear_Episode_1_C5143730001.partial.mp4.flv” 1>&2
RTMPDump v1.5a
(c) 2009 Andrej Stepanchuk, license: GPL
DEBUG: Parsing…
DEBUG: Parsed protocol: 3
DEBUG: Parsed host : streamflv.download.five.tv
DEBUG: Parsed app : a800/o10
DEBUG: Parsed playpath: mp4:bthttp/five/downloadstore/secure/fifthgear/movies/Fifth-Gear_S15_E1_Episode-1_FLVSTREAM.mp4?e=1241259330&h=709052a6c91019d9b7adb43eb1248a54
DEBUG: Setting buffer time to: 36000000ms
Connecting …
DEBUG: Protocol : RTMPE
DEBUG: Hostname : streamflv.download.five.tv
DEBUG: Port : 1935
DEBUG: Playpath : mp4:bthttp/five/downloadstore/secure/fifthgear/movies/Fifth-Gear_S15_E1_Episode-1_FLVSTREAM.mp4?e=1241259330&h=709052a6c91019d9b7adb43eb1248a54
DEBUG: tcUrl : rtmpe://streamflv.download.five.tv:1935/a800/o10
DEBUG: app : a800/o10
DEBUG: flashVer : LNX 9,0,124,0
DEBUG: live : no
DEBUG: timeout : 300 sec
DEBUG: SWFSHA256:
75 A4 A2 A2 DC 7E 7C 19 84 78 16 1B B1 C8 82 67 21 03 4F 4F 36 BE D7 D2 3C 6B F4 29 10 A5 00 A1
DEBUG: SWFSize : 848259
If I search for a show on Five like The Gadget Show (get_iplayer –type=five gadget), the results I get are the episodes from the latest series (series 11). Series 10 is also available on the website in flash but doesn’t show up in the results and Series 9 is available in Windows Media Player. Why isn’t series 10 indexed?
That is because I haven’t gotten around to doing older series for each programme. The older WMV versions aren’t supported by get_iplayer.
I should have mentioned that you can use: –pid five: to get any of those other flash based episodes.
Hi,
Great work on this. I’ve been using this since the 0.xx versions and it’s come so far.
Just the one question:
I use OSX (leopard) and is it possible to specify the location of rtmpdump and ffmpeg for getiplayer to remember? So that every time I put a ch4 or five download on, I don’t always have to use the commands –rtmpdump (location) –ffmpeg (location).
Using this method, it would remember where rtmpdump and ffmpeg is so we just have to put the download on and it will call for rtmpdump and ffmpeg in the locations which we have specified.
If that makes sense! Not sure I worded that very well!
Anyway, It’s a great program. Thanks for all the hard work you’ve put in to it.
Thanks,
Saf
You can use the –save option to save the current cmdline options (saved them in .get_iplayer/options)
Hi,
Thanks. I didn’t use that because I assumed that it would try to use rtmpdump and ffmpeg for the normal iphone downloads as well and always automatically download the flash version.
But it works perfect. Should’ve tried it first instead of assuming!
Thanks,
Saf
Is it me or if everytime I download something from CH4 I get ‘Countdown’… I’ve flushed the CH4 index
e.g.
./get_iplayer –pid=ch4:2872163
Should be Henry VIII’s Lost Palaces Episode 1
http://www.channel4.com/programmes/henry-viiis-lost-palaces/catch-up#2872163
Or The 100 Greatest Kids TV Shows
http://www.channel4.com/programmes/the-100-greatest-kids-tv-shows/catch-up#2872100
./get_iplayer –pid=ch4:2872100
More Countdown… any ideas
Countdown_s2009e70_2872100.mp4
I believe that this is fixed in version 1.76 (earlier today).
get_iplayer –update
should do it
Sorry, I meant version 1.77.
The update 1.80v doesnt seem to work with ch4 or five searchs anymore!
i get this message ERROR: Invalid type ‘five’ specified. Valid types are: itv,tv,podcast,radio
can anyone help?
I’ve moved get_iplayer to be plugin based. You just need to run the update again (i.e. ‘get_iplayer –update’ or get_iplayer update from the menu in Windows version). Failing that try ‘get_iplayer –update –force’ which will update the script and plugins regardless of the current version number.
This is a very minor thing, but you might want to reorder the numbering of CDNs for flashhigh and flashvhigh because akamai (corresponding to flashvhigh2 and flashhigh3) now definitely seems to be the default and indeed it has been the only flashvhigh or flashhigh CDN for all the programmes I’ve checked recently. (Of course I’m not suggesting that you remove the other CDNs, they might be used again for flashhigh or flashvhigh in the future. Interestingly though, flashaac1 (limelight) seems to be the preferred CDN for AAC versions of radio programmes.)
Also, I noticed that you’re currently using “http://www.bbc.co.uk/emp/9player.swf?revision=7978_8340” for the swfurl parameter for flashhd, flashvhigh and flashhigh and “http://www.bbc.co.uk/emp/9player.swf?revision=7276” for flashnormal, however the BBC website is currently using “http://www.bbc.co.uk/emp/9player.swf?revision=8812_8903”. I think if anyone looks at the log files it could look rather strange if requests are using a revision older than the website is using so you might want to update them.
swfurl for flashlow1 is currently set at “http://www.bbc.co.uk/emp/iplayer/7player.swf?revision=3897” (which returns a 404) and for flashlow2 at “http://www.bbc.co.uk/emp/9player.swf?revision=7978_8340”. I assume they both should use “http://www.bbc.co.uk/emp/9player.swf?revision=8812_8903”, but I don’t know how to access these streams on the BBC website so I’m not sure.
And for radio, flashaac1, flashaac2 and flashaudio also are using old revisions for swfurl.
Has anyone had any luck with the IT Crowd episode ‘Are We Not Men?’, with the PID 40439? All I get is ffmpeg spitting it back at me with:
[flv @ 0x1002400]negative cts, previous timestamps might be wrong
[NULL @ 0x1003400]error, non monotone timestamps 40 >= 40
av_interleaved_write_frame(): Error while opening file
WARNING: flv conversion failed – retaining flv file
All of the others work fine… any clues? :o)
Hi Andy, this programme does not seem to be available on the CH4 4OD website? I just tied it with the pid and am getting the default ‘Countdown’ – so no pid or bad pid?
Also that pid looks a little short, for a latter episodes of The IT Crowd programmes they are:
Calendar Geeks: 2873354
Friendface: 2872680
Tramps Like Us: 2872120
I can download at least one of these (2872120)
/get_iplayer –pid=ch4:2872120
Also are you using the latest version of get_iplayer (1.82 and also the latest rtmpdump 1.5a)?
Hi AudioBear,
Yes, I’m on get_iplayer v1.82 and rtmpdump v1.5a. Interestingly, the episode has vanished from the Channel 4 On Demand system now, so perhaps it was left on by mistake?
Thanks for looking into it, though!
Cheers,
Andy.
Actually Andy, I just looked at the video I downloaded and it is ‘Countdown’ again. Looks like a problem either with get_iplayer or CH4.
Phil, all the meta data is correct but the video is ‘Countdown’…
/get_iplayer –pid=ch4:2872120
I get the following error message when trying to search or download from ch4 or five. Using –type=ch4 or –type=five :
“Invalid file type five specified. Valid file types are: livetv, itv, tv, liveradio, podcast, radio.
I have the latest version of iplayer 2.11 and have updated plugins.
What am I doing wrong?
ch4, hulu and five are no longer supported.
OK, I read the update link at the top and understand. Hopefully these options will come back sometime.
Lee, for five at least rtmpdump is your friend 😉
Pages
Recent Posts
Categories