Encode of 54 sec video taking hours [Kaltura 2.0.1 on Amazon EC2 2 Core medium 32 bit ]

7 replies [Last post]
Joined: 07/10/2010
Points: 19

I've been testing uploading of video to a fresh install of kaltura on a 2 core EC@ instance (c1.medium 32 bit). Running on the prescribed ubuntu 10.04 with Lampp 1.7.1

All admin seems to be working fine - but encoding a test video clip is taking hours upon hours to complete.

Video details are:

54 second H264 encoded HD 1080i
About 52 MB large

Running two encoding profiles - one to Standard Small (H264), Standard Large (H264), HQ MP4 for download, HD (H264)

The other is Basic small -flv, and Standard large flv.

The first encode has been running for almost 3 hours. And this seems exceedingly long for a 54 second clip. Considering that I have hours of footage to upload, I'm not feeling to confident about relying on the conversion process on the server.

When I look at the logs of the converted videos I see this:

built on Jul 1 2009 11:44:09, gcc: 3.4.6 20060404 (Red Hat 3.4.6-10)
Compiler did not align stack variables. Libavcodec has been miscompiled
and may be very slow or crash. This is not a bug in libavcodec,
but in the compiler. You may try recompiling using gcc >= 4.2.
Do not report crashes to FFmpeg developers.

Which doesn't fill me with confidence....

Full log included below:
ajaxmac@ip-10-166-39-49:/opt/kalturaFiles/content/entry/data/4/267$ cat 0_3mqjlix7_0_lx4s8ppz_1.conv.log

----------------------
Executed by [ffmpeg] flavor params id [3]
----------------------

----------------------

----------------------

----------------------
/opt/kalturaServer/bin//ffmpeg -i /opt/kalturaFiles//content/entry/data/4/267/0_3mqjlix7_0_odkrblf0_1.mov -vcodec flv -b 1350k -s 1280x720 -r 25 -g 60 -acodec libmp3lame -ab 96k -ar 44100 -ac 2 -f flv -y /opt/kalturaServer/tmp//convert/convert_0_3mqjlix7_4c52a4775980f >> "/opt/kalturaServer/tmp//convert/convert_0_3mqjlix7_4c52a4775980f.log" 2>&1
----------------------

----------------------
-i __inFileName__ -vcodec flv -b 1350k -s 1280x720 -r 25 -g 60 -acodec libmp3lame -ab 96k -ar 44100 -ac 2 -f flv -y __outFileName__
----------------------

FFmpeg version 0.5, Copyright (c) 2000-2009 Fabrice Bellard, et al.
configuration: --arch=i386 --extra-cflags=-O2 --enable-bzlib --disable-devices --enable-libfaac --enable-libfaad --enable-libgsm --enable-libmp3lame --enable-libschroedinger --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-avfilter --enable-avfilter-lavf --enable-postproc --enable-swscale --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-optimizations --disable-stripping --shlibdir=/usr/lib --cpu=i386
libavutil 49.15. 0 / 49.15. 0
libavcodec 52.20. 0 / 52.20. 0
libavformat 52.31. 0 / 52.31. 0
libavdevice 52. 1. 0 / 52. 1. 0
libavfilter 0. 4. 0 / 0. 4. 0
libswscale 0. 7. 1 / 0. 7. 1
libpostproc 51. 2. 0 / 51. 2. 0
built on Jul 1 2009 11:44:09, gcc: 3.4.6 20060404 (Red Hat 3.4.6-10)
Compiler did not align stack variables. Libavcodec has been miscompiled
and may be very slow or crash. This is not a bug in libavcodec,
but in the compiler. You may try recompiling using gcc >= 4.2.
Do not report crashes to FFmpeg developers.

Seems stream 0 codec frame rate differs from container frame rate: 50.00 (50/1) -> 25.00 (25/1)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/opt/kalturaFiles//content/entry/data/4/267/0_3mqjlix7_0_odkrblf0_1.mov':
Duration: 00:00:53.82, start: 0.000000, bitrate: 7837 kb/s
Stream #0.0(eng): Video: h264, yuv420p, 1920x1080, 25 tbr, 25 tbn, 50 tbc
Stream #0.1(eng): Audio: aac, 44100 Hz, stereo, s16
Stream #0.2(eng): Data: tmcd / 0x64636D74
Output #0, flv, to '/opt/kalturaServer/tmp//convert/convert_0_3mqjlix7_4c52a4775980f':
Stream #0.0(eng): Video: flv, yuv420p, 1280x720, q=2-31, 1350 kb/s, 90k tbn, 25 tbc
Stream #0.1(eng): Audio: libmp3lame, 44100 Hz, stereo, s16, 96 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Press [q] to stop encoding
frame= 1345 fps= 23 q=31.0 Lsize= 9979kB time=53.80 bitrate=1519.5kbits/s
video:9295kB audio:631kB global headers:0kB muxing overhead 0.539080%

----------------------

----------------------

----------------------

----------------------

Joined: 02/22/2009
Points: 97

could you upload the source video so we can download it and try to understand if it is source-related ?

Joined: 07/10/2010
Points: 19

Hi : a copy of the video I was trying to upload and encode is here: http://www.wuala.com/ajaxmac/Documents/Ping/Ping%20Bash/Ping%20Bash%20vi...

Joined: 07/10/2010
Points: 19

I uploaded the same video to my kaltura.com account and it converted fine - to most flavours available in under an hour...

It must be a server setup issue... possibly related to that ffmpeg message...

However that is the default FFMPEG for Ubuntu 10.04, fetched through apt-get

Anyone else had upload issues on Ubuntu 10.04 with Kaltura?

Joined: 07/10/2010
Points: 19

OK

I did a fresh install on Ubuntu 10.04 64 bit, using an Amazon EC2 m1.large instance

Everything working as expected - created a Publisher account - edited back a test video to about 20 secs of HD, Quicktime H264 encoded.

Ran a couple of test uploads with different conversion profiles - and it seems that the server is running as expected.

It was probably the case that the previous server I was running was 32bit - and underpowered - i.e less cores, RAM etc.

I hope this helps someone else trying to set up their own Kaltura CE server

Joined: 01/05/2009
Points: 1697

Hi,

Can you please share the system details (specifications) of both of the old and the new system you used?
That can help others by learning base requirements...

Thanks!

Joined: 07/10/2010
Points: 19

Details of EC2 instance types are here:http://aws.amazon.com/ec2/#instance

Encoding working much faster on a Large Instance:
Large Instance 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of local instance storage, 64-bit platform

Than on the initial small instance:
Small Instance (Default) 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of local instance storage, 32-bit platform

Both Kaltura installs are/were on Ubuntu 10.04

Joined: 06/01/2010
Points: 72

Give our new AMIs a test:

http://www.kaltura.org/amazon-ec2-ami#comment-4097

Jason