Dec 25 2008

Merry Christmas!

Another great feature has been added to get_iplayer. You can now download programmes from the high quality flash version of the BBC iPlayer (640×360 resolution) using the optional rtmpdump tool. This means that you will be able to download ANY programme as soon as it is available on the iPlayer Flash web page.

Andrej Stepanchuk has very kindly written and released rtmpdump. If installed, get_iplayer (v1.04 or above) can use it by adding the –vmode=rtmp option to any TV download. This feature is currently an alpha test.
Pre-requisites are:

  • Installation of rtmpdump
  • ffmpeg (to convert the flv file into mp4 format)

You can get rtmpdump here (compiled for Linux/x86-32bit, MacOSX/x86 and cygwin). The Home Page and C++ source is here. I recommend you compile your own – it is very easy – just make sure you have the boost libraries installed.

In get_iplayer you’ll need to specify the location of rtmpdump binary (using –rtmpdump) if it is not already in the path.

Let me know how it goes.

Update (28th December 2008)

Andrej has released rtmpdump-v1.1 this addresses issues with 64-bit Linux and adds download resume support. Use in conjunction with the latest get_iplayer 1.05 or newer.

If you come across problems specifically related to rtmpdump (i.e. not get_iplayer) you can post them at the Sourceforge page for rtmpdump.

Update (2nd January 2009)

For get_iplayer 1.09 and above to use the RTMP features you’ll need rtmpdump-v1.2+ this addresses issues with Normal quality RTMP Flash downloads and adds download resume support for this quality.

52 Comments

  • On December 27th, 2008, David said:

    Hi
    Did not seem to work. Using 64bit ubuntu gutsy
    INFO: Getting media stream metadata for The Andrew Marr Show – 21/12/2008, b00gbc4r
    INFO: RTMP high quality stream URL: rtmp://bbciplayertokfs.fplive.net:1935/ondemand?_fcs_vhost=bbciplayertokfs.fplive.net&auth=b6cf8746b553d582d613a9705cbc823b&aifp=v001&slist=mp4:b000gbc4rH264100906696
    INFO: File name prefix = The_Andrew_Marr_Show_-_21_12_2008_b00gbc6r_default
    INFO: Getting version pids for programme b00gbc6r
    RTMP Dumper v1.0
    (c) 2009 Andrej Stepanchuk, license: GPL

    Connecting to rtmp://bbciplayertokfs.fplive.net:1935/ondemand?_fcs_vhost=bbciplayertokfs.fplive.net&auth=b6cf8746b553d582d613a9705cbc823b&aifp=v001&slist=mp4:b000gbc4rH264100906696 …
    DEBUG: Hostname: bbciplayertokfs.fplive.net
    DEBUG: Port: 1935
    DEBUG: Auth: b6cf8746b553d582d613a9705cbc823b0
    DEBUG: connected, hand shake:
    DEBUG: handshaked
    Connected…

    DEBUG: GetNextMediaPacket, received: server BW
    DEBUG: GetNextMediaPacket, received: client BW
    DEBUG: server sent ping. type: 0
    DEBUG: GetNextMediaPacket, received: invoke
    DEBUG: Decode – unknown datatype 0x06
    ERROR: HandleInvoke, error decoding invoke packet
    DEBUG: GetNextMediaPacket, received: invoke
    DEBUG: START AMF Object Dump:
    DEBUG: Property:
    DEBUG: Property:
    DEBUG: END AMF Object Dump:
    DEBUG: HandleInvoke, server invoking
    ERROR: rtmp server requested close
    Closing connection… done!

    FFmpeg version SVN-rUNKNOWN, Copyright (c) 2000-2007 Fabrice Bellard, et al.
    configuration: –enable-gpl –enable-pp –enable-swscaler –enable-pthreads –enable-libvorbis –enable-libtheora –enable-libogg –enable-libgsm –enable-dc1394 –disable-debug –enable-libmp3lame –enable-libfaadbin –enable-libfaad –enable-libfaac –enable-xvid –enable-x264 –enable-liba52 –enable-amr_nb –enable-amr_wb –enable-shared –prefix=/usr
    libavutil version: 1d.49.3.0
    libavcodec version: 1d.51.38.0
    libavformat version: 1d.51.10.0
    built on Aug 10 2008 09:17:28, gcc: 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)
    ./The_Andrew_Marr_Show_-_21_12_2008_b00gbc6r_default.partial.mov.flv: Unknown format
    ERROR: flv to mp4 conversion failed

    The .flv file was empty

    • On December 27th, 2008, linuxcentre said:

      @David
      Did you try compiling rtmpdump from source? Or are you using my 32-bit compiled version?

      • On December 27th, 2008, David said:

        I did compile from source, did not seem to give any errors

        if I run your 32bit executable I get
        rtmpdump: error while loading shared libraries: libboost_regex.so.3: cannot open shared object file: No such file or directory
        would it be worth looking for the 32bit libraries?

        • On December 27th, 2008, Alex said:

          How do I compile it for windows?

          • On December 27th, 2008, linuxcentre said:

            @Alex,
            I imagine the boost libraries and gcc/c++ compiler should be available under cygwin. No idea how to do this for native Windows without cygwin though.

            • On December 27th, 2008, linuxcentre said:

              @David
              Yes, you’ll need the boost libraries and c++ compiler for ubuntu 64-bit then just run ‘make’ in the source directory of rtmpdump. I’ve not tried this under 64bit Linux yet – I don’t even have access to such high-tech machinery!

              • On December 27th, 2008, David said:

                Hi
                My compiled 64bit version of rtmpdump does not seem to work.
                Have found 32bit libraries for boost so using your compiled rtmpdump now downloads a .flv
                file. it sticks near the end and times out after a few minutes. Then ffmpeg cant seem to cope
                I dont know whether that is because I need a newer version of ffmpeg or the flv file
                was not down loaded properly.
                I’ll have to give up for the time being
                64 bits is a nightmare for media things!
                Thanks for your help anyway

                it said

                …….
                WARNING: ignoring too small audio packet: size: 0
                DEBUG: GetNextMediaPacket, received: notify
                DEBUG: GetNextMediaPacket, received: notify
                WARNING: ignoring too small video packet: size: 5
                WARNING: ignoring too small video packet: size: 5
                WARNING: ignoring too small audio packet: size: 0
                DEBUG: GetNextMediaPacket, received: notify
                DEBUG: server sent ping. type: 1
                DEBUG: GetNextMediaPacket, received: invoke
                DEBUG: START AMF Object Dump:
                DEBUG: Property:
                DEBUG: Property:
                DEBUG: Property: NULL
                DEBUG: Property: OBJECT ====>
                DEBUG: START AMF Object Dump:
                DEBUG: Property:
                DEBUG: Property:
                DEBUG: Property:
                DEBUG: Property:
                DEBUG: Property:
                DEBUG: Property:
                DEBUG: END AMF Object Dump:
                DEBUG: END AMF Object Dump:
                DEBUG: HandleInvoke, server invoking
                DEBUG: HandleInvoke, onStatus: NetStream.Play.Stop
                Closing connection… done!

                FFmpeg version SVN-r14255, Copyright (c) 2000-2008 Fabrice Bellard, et al.
                configuration: –enable-x11grab –enable-gpl
                libavutil version: 49.7.0
                libavcodec version: 51.60.0
                libavformat version: 52.17.0
                libavdevice version: 52.0.0
                built on Jul 16 2008 16:20:46, gcc: 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)
                [h264 @ 0xd17790]brainfart cropping not supported, this could look slightly wrong …
                Input #0, flv, from ‘./The_National_Lottery_Euromillions_Draw_-_26_12_2008_b00g7skj_default.partial.mov.flv’:
                Duration: 00:03:17.20, start: 0.000000, bitrate: N/A
                Stream #0.0: Video: h264, yuv420p, 640×360 [PAR 90:90 DAR 16:9], 25.00 tb(r)
                Stream #0.1: Audio: 0x0000, 44100 Hz, stereo
                Output #0, mp4, to ‘./The_National_Lottery_Euromillions_Draw_-_26_12_2008_b00g7skj_default.partial.mov’:
                Stream #0.0: Video: 0x0000, yuv420p, 640×360, q=2-31, 1000.00 tb(c)
                Stream #0.1: Audio: 0x0000, 44100 Hz, stereo
                Stream mapping:
                Stream #0.0 -> #0.0
                Stream #0.1 -> #0.1
                [mp4 @ 0xd18070]track 1: codec frame size is not set
                Could not write header for output file #0 (incorrect codec parameters ?)

                • On December 28th, 2008, Bigaules said:

                  Ubuntu 32 bit. This is as far as I get.

                  error while loading shared libraries: libboost_regex.so.3: cannot open shared object file: No such file or directory

                  • On December 28th, 2008, linuxcentre said:

                    @Bigaules, David
                    See the update in the post above… Updated rtmpdump is out – fixed 64-bit Linux issues – also recommend compiling your own rtmpdump.

                    • On December 28th, 2008, David said:

                      Hi
                      Pleased to say new compiled 64bit version of rtmpdump works
                      and with a new version of ffmpeg everything works as expected
                      Thanks for your help

                      • On December 28th, 2008, Bigaules said:

                        @Phil

                        I ran these commands to get the boost library

                        sudo apt-get install libboost-thread-dev

                        sudo apt-get install build-essential

                        when I compile I get this error

                        g++ rtmp.cpp log.cpp AMFObject.cpp rtmppacket.cpp rtmpdump.cpp -o rtmpdump -lboost_regex
                        rtmpdump.cpp: In function ‘int main(int, char**)’:
                        rtmpdump.cpp:514: warning: unknown conversion type character ‘)’ in format

                        but the flv file still downloads ok. Is the compile error of any significance?

                        • On December 28th, 2008, Andrej said:

                          @Bigaules

                          Hi, this is a small bug in my rtmpdump, fixed it, thanks 🙂 (I will upload the new version shortly)

                          • On December 29th, 2008, user said:

                            Thanks for this! Since the BBC iplayer Desktop for linux doesn’t seem to be working properly, I was glad to find this software, also glad of these higher quality downloads.

                            Worked fine for me on 32bit linux, just had ffmpeg issues on 64bit linux similar to David above, so I’ll try updating that.

                            A question, is it possible to just keep the flv format download instead of converting it to a .mov at the end? (doesn’t it result in a loss of quality performing such a conversion?)

                            • On December 29th, 2008, linuxcentre said:

                              @user,
                              Good idea, I may well add that option in.
                              There is no loss of quality – all that ffmpeg should be doing is changing the container format – that is why it does it so fast.

                              • On December 30th, 2008, Andrej said:

                                I personally think that the flv format is a nightmare since it is ONLY supported by Adobe’s software completely. Only recently players like mplayer implement it to a sufficicient extent such that it becomes almost useful (e.g. support for the new h264/avc streams in flv was added not so long ago). The best example is: my rtmpdump has to omit initialization video/audio packets which don’t contain data since ffmpeg and all other implementations get confused (they think the stream does not contain any codec prameters since they read these initialization packets which are not the actual stream).

                                It is really the best not to keep any flv files, but put the contained data streams into a widely a standard container like mp4. FLV is not a reasonable media format! (its a Adobe’s stream container for designed for the use with streaming server software. Keeping FLV files is equivalent to keeping network TCP dumps as your video files).

                                • On December 30th, 2008, Andrew said:

                                  Can you please publish some more detailed instructions of how to install/compile rtmpdump under Ubuntu?

                                  I have tried this:

                                  1) sudo apt-get install libboost-thread-dev
                                  2) sudo apt-get install build-essential
                                  3) Downloaded the latest rtmpdump from sourceforge
                                  4) Uncompressed and ran “make”
                                  5) Moved the rtmpdump executable to /home/myid/.rtmpdump

                                  When I got to stage 4 make had this error message:
                                  g++ rtmp.cpp log.cpp AMFObject.cpp rtmppacket.cpp rtmpdump.cpp -o rtmpdump -lboost_regex
                                  rtmpdump.cpp: In function ‘int main(int, char**)’:
                                  rtmpdump.cpp:514: warning: unknown conversion type character ‘)’ in format

                                  When I try to download something with get_iplayer I get the .mov.flv file fine but get_iplayer then seems to “hang” for a while before creating a zero-byte size .mov file.

                                  • On December 30th, 2008, Andrej said:

                                    This warning is due to a typo in the source code and does not matter. Since you compiled rtmpdump it should work. You should see the number of downloaded bytes (percentage) increase. Check the size of the partial….flv file, it should increase as well while you download. Check that you use the latest ffmpeg since old versions of ffmpeg don’t support flv!!

                                    It is known that rtmpdump hangs in the end because it waits for a play.stop event from the server to make sure you got all of the stream (so far I have no plans/ideas how to fix that, just press Ctrl-C if you’re convinced the download is finished and use ffmpeg manually if necessary).

                                    You can post rtmpdump output here if you want me to have a closer look.

                                    • On December 30th, 2008, Bigarules said:

                                      @Andrew

                                      You need to compile ffmpeg as the version in the Ubuntu 8.10 repository won’t work.

                                      There are some clear instructions here

                                      http://ubuntuforums.org/showthread.php?t=786095

                                      • On December 30th, 2008, linuxcentre said:

                                        @user,
                                        See Andrej’s response above for a sensible answerre flv. I have however extended –raw option in get_iplayer v1.07 so that the ffmpeg conversion is not run should you so want to keep the flv stream for any reason.

                                        • On January 2nd, 2009, Takla said:

                                          I can’t tell from your description if the latest version of get_iplayer includes rtmpdump or if rtmpdump is an extra or if maybe compiling rtmpdump provides rtmpdump and another version of get_iplayer . Words like integration and merge and provide and conjunction are truly horrible if not used very clearly indeed. I even tried reading it all again while sober but it was no clearer.

                                          • On January 3rd, 2009, linuxcentre said:

                                            rtmpdump is a prerequisite. I’ve reworded it more carefully for you 🙂 Have another read.

                                          • On January 4th, 2009, Takla said:

                                            thanks 🙂

                                            • On January 4th, 2009, radio_listener said:

                                              Let me first thank you for this very neat script and also for the high quality rmtpdump.

                                              Using get_iplayer v1.1 I am not able to download the flashhigh version although the iphone version is working with the very same proxy.

                                              ./rtmpdump/get_iplayer_current –get b0074dkr –subtitles –vmode=flashhigh -p http://212.241.180.239:81 –rtmpdump=./rtmpdump/rtmpdump -v

                                              Output is:
                                              http://www.pastie.org/352186

                                              • On January 4th, 2009, radio_listener said:

                                                Tried the different rtmp versions.
                                                Only flashwii is working.

                                                • On January 4th, 2009, linuxcentre said:

                                                  I did some investigations on this. The authstring given by the BBC server is only valid for a number of seconds. Now, if you use the –subtitles option, the subtitles are downloaded between when the authstring is determined and whent he rtmpdump command starts. If the subtitles download takes too long then the rtmp download fails. I’ll try to fix that in v1.11 but try not using –subtitles and see if it helps. Please let me know how you get on.

                                                  • On January 4th, 2009, linuxcentre said:

                                                    It would seem that if get_iplayer doesn’t get to the rtmpdump stage by around 36 seconds then it will fail.

                                                • On January 4th, 2009, user said:

                                                  Thanks for the replys regarding the FLV format, I now accept that the Quicktime .mov is a better way to keep the videos.

                                                  Also, it is working fine on 64bit openSUSE 11.0 using up-to -date ffmpeg (packman), and needed the boost-devel package to compile rtmpdump

                                                  One last question: Is there a reason why the downloaded .mov files play fine on linux, but not when I copy them to a Mac (PPC, Quicktime 7.5)?

                                                  • On January 4th, 2009, linuxcentre said:

                                                    If you are referring to the mov files downloaded in the default mode (i.e. iphone/h.264) then coincidentally I’ve today been working on get_iplayer v1.11 which may well fix that. It might be due to the mov file being non-compliant (the first atom was not ftyp before). Please let me know if v1.11 helps.
                                                    If however yo are referring to the rtmp downloaded mov/mp4 files having probs with MacOSX then let me know…

                                                  • On January 7th, 2009, user said:

                                                    Hi, I had been referring to the rtmp downloads having trouble with Mac OSX, however it seems it was just a file extension issue. Files named with .mov are not recognised, while ones with .mp4 are. I notice that .mp4 is the extension given by default now.
                                                    Cheers!

                                                    • On January 12th, 2009, user said:

                                                      I have noticed problems with rtmpdump v1.3 using current get_iplayer (v1.15 – v1.18)

                                                      When I revert to rtmpdump 1.2a it works again! Not sure if that is a problem with rtmpdump or get_iplayer.

                                                      Just mentioning it in case it helps someone.

                                                      • On January 12th, 2009, linuxcentre said:

                                                        Hi, Can you be more specific?

                                                        • On January 12th, 2009, linuxcentre said:

                                                          Ahh, I see, video download hangs in flashhigh mode… Yes thanks best use rtmpdump-1.2a for now.

                                                        • On January 16th, 2009, richard said:

                                                          Can you enlighten me as to which boost libraries I need to install?

                                                          Ubuntu 8.04
                                                          rtmpdump v1.3

                                                          From earlier posts installed: libboost-thread-dev and dependencies
                                                          Ran make and got this error:

                                                          richard@base-desktop:/usr/local/src/rtmpdump$ make
                                                          g++ -Wall rtmp.cpp log.cpp AMFObject.cpp rtmppacket.cpp rtmpdump.cpp -o rtmpdump -lboost_regex
                                                          rtmp.cpp:30:27: error: boost/regex.hpp: No such file or directory
                                                          rtmp.cpp: In member function ‘bool RTMP_LIB::CRTMP::Connect(char*, char*, char*, char*, char*, char*, char*, double)’:
                                                          rtmp.cpp:80: error: ‘boost’ has not been declared
                                                          rtmp.cpp:80: error: expected `;’ before ‘re’
                                                          rtmp.cpp:82: error: ‘boost’ has not been declared
                                                          rtmp.cpp:82: error: ‘re’ was not declared in this scope
                                                          rtmp.cpp:97: error: ‘boost’ has not been declared
                                                          rtmp.cpp:97: error: expected `;’ before ‘matches’
                                                          rtmp.cpp:99: error: ‘boost’ has not been declared
                                                          rtmp.cpp:99: error: expected `;’ before ‘re1’
                                                          rtmp.cpp:100: error: ‘boost’ has not been declared
                                                          rtmp.cpp:100: error: ‘matches’ was not declared in this scope
                                                          rtmp.cpp:100: error: ‘re1’ was not declared in this scope
                                                          rtmp.cpp:109: error: ‘matches’ was not declared in this scope
                                                          rtmp.cpp:113: error: ‘matches’ was not declared in this scope
                                                          make: *** [rtmpdump] Error 1

                                                          Installed: libboost regex libboost filesystem

                                                          Ran make again. Same error.

                                                          • On January 16th, 2009, richard said:

                                                            Problem sorted.

                                                            Installed: libboost-regex-dev

                                                            Ran make. No errors.

                                                          • On January 17th, 2009, eldavio said:

                                                            Has anyone compiled a working rtmpdump for Mac OS X?

                                                            I fiddled with the makefile and got it to compile, using MacPorts versions of the boost libraries.

                                                            However, when I try to download video it fails every time reporting a bus error.

                                                            This is under 10.5 on a MacBook Pro. I compiled against boost 1.35

                                                            Any hints, or is this just completely unsupported presently?

                                                            • On January 17th, 2009, linuxcentre said:

                                                              I have no idea about this. Have you tried asking at the rtmpdump sourceforge page?

                                                              • On January 18th, 2009, Uldis Bojars said:

                                                                Try replacing times(0) with time(0) in the rtmp.cpp file.
                                                                I am not sure if that is a proper fix, but it gets rid of the bus error on a Mac.

                                                                Has anyone tried an succeeded in using rtmpdump for downloading videos from videolectures.net?

                                                                • On February 2nd, 2009, linuxcentre said:

                                                                  rtmpdump MacOSX binary is now on the get_iplayer download page

                                                                  • On March 25th, 2009, ernesto said:

                                                                    i would need a ppc binary for osx… can you help me?

                                                                    • On March 26th, 2009, linuxcentre said:

                                                                      Anyone? – I don’t have access to this platform…

                                                                      • On March 26th, 2009, linuxcentre said:

                                                                        Actually, if you compile it yourself you could email me the binary and I’ll put it up here.

                                                                  • On January 17th, 2009, Andrej said:

                                                                    @user
                                                                    Please use rtmpdump v1.3a, the 1.3 contains a nasty bug and will hang during a flashhhigh download (I forgot to comment out some wrong debug code, fixed that in v1.3a).

                                                                    • On January 28th, 2009, Takla said:

                                                                      rtmpdump 1.3b and get_iplayer have been outstanding. I had some problems earlier with rtmpdump 1.2 and 1.3, compiled the newest rtmpdump and had kind of forgotteten about using it. Then I unwittingly asked get_iplayer to download a show which had just been broadcast and instead of the not ready for download message I instead had a very nice quality mp4 arrive. Ooooooh! Excellent, thanks!

                                                                      • On February 2nd, 2009, j said:

                                                                        Some of us simply cannot get rtmpdump working due to compiler/boost issues. Can anyone post a CygWin or Intel Mac binary?

                                                                        • On February 2nd, 2009, Richard said:

                                                                          Hi,

                                                                          You can get a Mac binary from my site – http://www.smallsoftware.co.uk/downloads

                                                                          Richard

                                                                          • On February 21st, 2009, Brian said:

                                                                            Anybody had any luck building for PPC? I see Intel-only. 🙁

                                                                          • On February 2nd, 2009, linuxcentre said:

                                                                            I’ve managed to compile rtmpdump.exe for cygwin windows 32bit. You can find it on the download page together with all the links to other support progrms.

                                                                          • On February 2nd, 2009, j said:

                                                                            Richard: a million thanks. This makes things so much easier!

                                                                            • On March 2nd, 2009, Rolf said:

                                                                              Richard- Great post. I was able to get a Win32 version of rtmpdump 1.3d compiled and running in VS2008, no cygwin needed. I’m not that great at C++, so there could be bugs. That said, if you think it would be helpful for others, I can send it to you to post.

                                                                              • On March 2nd, 2009, linuxcentre said:

                                                                                That’s great! I think you should send the patch (if any) or build instructions to the rtmpdump author so he can add it to his release notes/build instructions. I will not be able to build this myself and I will put it on the download page if someone is willing to maintain new builds of it 🙂

                                                                              • 2 old comments are not displayed. Click to display all comments

                                                                              WordPress Themes