Kaltura CE 3.0 to 4.0 Migration Thread...

16 replies [Last post]

Thu, 07/21/2011 - 06:46

rethinkweb

Joined: 11/17/2010

Points: 46

This thread is intended for discussion, documentation and testing for CE 3.0 to 4.0 migration...

Feel free to post if you have thoughts to share...

Thu, 07/21/2011 - 16:24

Kalturian

Joined: 01/05/2009

Points: 1697

Upgrading from Cassiopeia (CE 3.0) to Dragonfly (CE 4.0)

This is an experimental upgrade flow from Kaltura CE version 3 (aka "Cassiopeia") to 4 (aka "Dragonfly").
The scripts and steps below were not tested. It is provided with no warrenty and is not guaranteed to work smoothly.
This was created based on clean environment assumption (no changes made to the code or applications, no new applications installed).
Please use with care and let us know by commenting this thread if you have any questions, fixes or suggestions.

Export your CE Cassiopeia (3.0) DataBase

Run the following command (replace [user] and [password] with relevant values):
mysqldump kaltura -u[user] -p[password] > kaltura.sql

Install CE Dragonfly (4.0)

If you wish to install CE Dragonfly (4.0) on the same machine CE Cassiopeia is installed you have to backup the Cassiopeia installation data.
Assuming clean evnironment (no changes made to the code or applications, no new applications installed) you need to backup Cassiopeia DB (which you already did in the previous step) and your web dir where all your content is saved (just copy this folder).
The CE Dragonfly installation will detect leftovers from Cassiopeia installation, In order to continue with Dragonfly installation these leftovers will be removed by the installer.

Install Kaltura CE 4.0 in a new folder.
Do not install in the same folder as CE 3.0 was installed on.

Import your Cassiopeia DataBase

Run the following command (replace [user] and [password] with relevant values):
mysql kaltura -u[user] -p[password] < kaltura.sql

Rsync Cassiopeia web directory to your Dragonfly web directory

For example, if the Cassiopeia web dir is in /opt/kalturaC/web you should run - rsync /opt/kalturaC/web/* /opt/kalturaD/web/
Note that you should use rsync and not cp, even in the same server

Run the migration scripts

Run all sql scripts from /opt/kaltura/app/deployment/updates/sql
Run all php scritps from /opt/kaltura/app/deployment/updates/scripts with realrun param (for example: php script_name.php realrun)

Deploy the KMC uiConf

Deploy kmc v4.0.12.5 ui conf by running:
php /opt/kaltura/app/deployment/uiconf/deploy_v2.php –ini=/opt/kaltura/web/flash/kmc/v4.0.12.5/config.ini

Mon, 04/02/2012 - 08:03

ohmalone

Joined: 10/29/2010

Points: 23

i don't understand step to migration CE 3.0 to CE 4.0
- Run the migration scripts
What is command for run sql script?
What is database for running sql script?
- Deploy the KMC uiConf
1. php /opt/kaltura/app/deployment/uiconf/deploy_v2.php –ini=/opt/kaltura/web/flash/kmc/v4.0.12.5/config.ini
Where is dir 'v4.0.12.5' i don't see dir in the path?

Best Regards

Mon, 02/20/2012 - 11:34

neuron005

Joined: 10/05/2011

Points: 103

Thanks for this post
Does this work for CE5 too ? I want to restore my CE5 machine.I backed up my kaltura database in CE5 and imported it after installation, it did not work.Is there something related to session with entries? If so, how will I perform this restore?
Please post a reply
Thanks in advance

Wed, 02/29/2012 - 16:34

Kalturian

Joined: 01/05/2009

Points: 1697

Hi guys,

This wasn't tested thoroughly, but find below the general steps to upgrade 4 to 5.
The workflow is basically the same – install eagle, migrate DF db, run upgrade scripts, recreated sphinx, deploy ui conf, upgrade DWH, copy web dir.

In Eagle there is an update script /opt/Kaltura/app/deployment/updates/update.php that runs all the upgrade scripts from /opt/Kaltura/app/deployment/updates/ dir (sql and php) so there is no need to run them one by one
In Eagle the upgrade script was added to the installer package – after Eagle is installed the script is located at /opt/Kaltura/app/dwh/ddl/migrations/20110927_dragonfly_to_eagle
The KMC version in Eagle is v4.2.14.9 so to deploy Eagle UI conf we need to replace step 10 with this command:
php /opt/kaltura/app/deployment/uiconf/deploy_v2.php --ini=/opt/kaltura/web/flash/kmc/v4.2.14.9/config.ini

Mon, 03/05/2012 - 09:49

neuron005

Joined: 10/05/2011

Points: 103

Thanks
I applied the same for CE5 to CE5. And I had to execute this script also with partner id param
php /opt/kaltura/app/scripts/utils/updatePartnerEntries2Sphinx.php “partner_id".
Now I can see videos in content tab and can play them. But I can not upload videos. kaltura_api_v3.log is throwing

2012-03-05 04:48:11 [107.21.228.222] [446603975] [API] [KalturaFrontController->getExceptionObject] ERR: exception 'KalturaAPIException' with message 'Invalid KS [MWFlMTljZjk4Y2FhYjQ4MWU3NjA2YzlhZDAzNWI3ZDRiNzU3NGQ1N3wtMTs7MTMzMzUzMjg5MTsyOzEzMzA5NDA4OTEuNzk0NTtiYXRjaFVzZXI7Oy0xOw==]. Error [-1,INVALID_STR]' in /opt/kaltura/app/alpha/apps/kaltura/lib/kCurrentContext.class.php:144

Stack trace:

#0 /opt/kaltura/app/api_v3/lib/KalturaDispatcher.php(76): kCurrentContext::initKsPartnerUser('MWFlMTljZjk4Y2F...', '-1', '')

#1 /opt/kaltura/app/api_v3/lib/KalturaFrontController.php(97): KalturaDispatcher->dispatch('dropfolder_drop...', 'list', Array)

#2 /opt/kaltura/app/api_v3/web/index.php(19): KalturaFrontController->run()

#3 {main}

I think there is something related with sessions. Can you help me out?
Thanks in advance

Wed, 03/07/2012 - 12:38

neuron005

Joined: 10/05/2011

Points: 103

Hiii

I resolved the above error by replacing KS values of batch partner in DB backup with the new installed DB(simply replaced old with new one). So the above error stopped. Now I have all of my previous videos and can play them. But I can not upload new videos. It is always in queue. I tried to upload video by using JAVA API and I WAS SUCCESSFUL. So, what could be the possible error ?

Tue, 03/06/2012 - 14:39

neuron005

Joined: 10/05/2011

Points: 103

The upgradation from CE 4 to CE 5 with the above described steps was successful and the videos were playing fine.

But it introduced a new problem that no new videos could be added. I was able to upload the videos but it got stuck in the converting process. The batch process control was showing the files but they were not processed

The log files are attached below.

1.) kaltura api log
2.) batch convert log

Thanks in advance.

Attachment	Size
convert.txt	3.25 KB
kaltura_api_v3.txt	5.36 KB

Fri, 03/30/2012 - 21:16

chris.kloosterman

Joined: 03/10/2011

Points: 8

We were running Kaltura 3.0.0 on a Debian 6 64-bit server. Here are the steps we followed to upgrade to 4.0.0 (mostly successfully):

Follow this procedure: http://www.kaltura.org/kaltura-ce-30-40-migration-thread

*except* you need to move /opt/kaltura out of the way and install the new version to /opt/kaltura too (assuming that's where the old version was installed to). Plus, you need to add the following lines to /opt/kaltura/app/deployment/bootstrap.php after the KAutoloader::register(); call:

require_once(ROOT_DIR . '/admin_console/lib/Kaltura/AdminConsoleUserPartnerData.php');
require_once(ROOT_DIR . '/api_v3/lib/KalturaServiceConfig.php');
require_once(ROOT_DIR . '/api_v3/lib/KalturaServicesMap.php');

... because these classes didn't work properly in the migration scripts for some reason.

The script 2011-03-27_flavor_params_iphone_ipad_data_update.sql needs to be edited in a text editor. You need to take out the commented rows and add semicolons to each update statement.

Fixes necessary after running the SQL and PHP scripts:

sudo chown www-data:www-data -R /opt/kaltura/app/cache
mysql -u root -p kaltura

update partner set admin_secret='(new secret pulled from /opt/kaltura/app/batch/batch_config.ini)' WHERE id=-1;
exit;

We use 4 threads for video conversion rather than 2, so we also did this step:

cd /opt/kaltura/app/batch
sudo vi batch_config.ini

maxInstances = 4 (for KAsyncConvert section)

Our sphinx wasn't starting up correctly, so we fixed that by doing this:

sudo mkdir -p /usr/local/var/data
sudo update-rc.d sphinx_watch.sh defaults
sudo /etc/init.d/sphinx_watch.sh start

For some reason our kConf.php was referencing different versions of software than we had available.

sudo vi /opt/kaltura/app/alpha/config/kConf.php (update versions of kmc software to the ones available in /opt/kaltura/web/flash/kmc)

Wait until the next 30 minute cron interval so that batch processes start up correctly, then test thoroughly.

Mon, 04/02/2012 - 08:00

ohmalone

Joined: 10/29/2010

Points: 23

To chris.kloosterman

"The script 2011-03-27_flavor_params_iphone_ipad_data_update.sql needs to be edited in a text editor. You need to take out the commented rows and add semicolons to each update statement."

i'm editing the file "2011-03-27_flavor_params_iphone_ipad_data_update.sql" but not sure to editing follow with instructions
and i upload file then and editing instructions
Can you check my file please?

Best regards

Attachment	Size
2011-03-27_flavor_params_iphone_ipad_data_update.txt	3.45 KB

Mon, 04/02/2012 - 17:47

#10

chris.kloosterman

Joined: 03/10/2011

Points: 8

Try the attached script. I've removed all commented lines (starting with '#') and have also added the semicolons in the correct spots (at the end of each update SQL statement).

Attachment	Size
2011-03-27_flavor_params_iphone_ipad_data_update.txt	3.26 KB

Tue, 04/03/2012 - 03:58

#11

ohmalone

Joined: 10/29/2010

Points: 23

Thank you so much for the answer.
But i'm using Ubuntu Server 10.04 LTS. and using follow procedure: http://www.kaltura.org/kaltura-ce-30-40-migration-thread
and the Last "Deploy the KMC uiConf" Do you running the command? (i'm not running command)
But i doing follow the step of you to

" Our sphinx wasn't starting up correctly, so we fixed that by doing this:
sudo mkdir -p /usr/local/var/data
sudo update-rc.d sphinx_watch.sh defaults
sudo /etc/init.d/sphinx_watch.sh start "

after i'm running command it display the following message

"kaltura@kaltura4:/opt/kaltura/app/scripts$ sudo update-rc.d sphinx_watch.sh defaults
update-rc.d: warning: /etc/init.d/sphinx_watch.sh missing LSB information
update-rc.d: see
Adding system startup for /etc/init.d/sphinx_watch.sh ...
/etc/rc0.d/K20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc1.d/K20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc6.d/K20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc2.d/S20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc3.d/S20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc4.d/S20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc5.d/S20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
kaltura@kaltura4:/opt/kaltura/app/scripts$ sudo /etc/init.d/sphinx_watch.sh start
Starting Sphinx Watch Daemon: NOT OK"

Can you help me please?
and i'm trying to login KMC ui it appear message "Wrong password supplied".
PS: i'm editing kConf.php complete
- kmc_content_version
- kmc_account_version
- kmc_appstudio_version
- kmc_dashboard_version
- kmc_login_version

Tue, 04/10/2012 - 23:42

#12

chris.kloosterman

Joined: 03/10/2011

Points: 8

> But i'm using Ubuntu Server 10.04 LTS. and using follow procedure: http://www.kaltura.org/kaltura-ce-30-40-migration-thread
> and the Last "Deploy the KMC uiConf" Do you running the command? (i'm not running command)

Yes, I did run that command. Just run the whole line as one command at the console, substituting a different directory if necessary.

> But i doing follow the step of you to

" Our sphinx wasn't starting up correctly, so we fixed that by doing this:
sudo mkdir -p /usr/local/var/data
sudo update-rc.d sphinx_watch.sh defaults
sudo /etc/init.d/sphinx_watch.sh start "

> after i'm running command it display the following message

> "kaltura@kaltura4:/opt/kaltura/app/scripts$ sudo update-rc.d sphinx_watch.sh defaults
update-rc.d: warning: /etc/init.d/sphinx_watch.sh missing LSB information
update-rc.d: see
Adding system startup for /etc/init.d/sphinx_watch.sh ...
/etc/rc0.d/K20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc1.d/K20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc6.d/K20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc2.d/S20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc3.d/S20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc4.d/S20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
/etc/rc5.d/S20sphinx_watch.sh -> ../init.d/sphinx_watch.sh
kaltura@kaltura4:/opt/kaltura/app/scripts$ sudo /etc/init.d/sphinx_watch.sh start
Starting Sphinx Watch Daemon: NOT OK"

The first thing (missing LSB information) is fine. If you want to, you can add the following LSB information to the top of the file (after the #! line):

### BEGIN INIT INFO
# Provides: sphinx-watch
# Required-Start: $syslog $remote_fs $network
# Required-Stop: $syslog $remote_fs $network
# Default-Start: 2 3 4 5
# Default-Stop: 1
# Short-Description: Sphinx Watch
### END INIT INFO

As for sphinx not starting up correctly, you're on your own there. I didn't run into that problem. Check your ps -ef output to see if there's a searchd running, and if not, check in /opt/kaltura/log to see if anything helps you there.

> and i'm trying to login KMC ui it appear message "Wrong password supplied".

You have to log in with your old username and password (from Kaltura3) not the username and password you supplied when installing Kaltura4.

> PS: i'm editing kConf.php complete
- kmc_content_version
- kmc_account_version
- kmc_appstudio_version
- kmc_dashboard_version
- kmc_login_version

I think these are the settings I had to change:

"kmc_content_version" => 'v3.2.11.1',
"kmc_account_version" => 'v3.1.1',
"kmc_appstudio_version" => 'v2.2.3',
"kmc_rna_version" => 'v1.1.5',
"kmc_dashboard_version" => 'v1.0.14',

Wed, 04/11/2012 - 07:51

#13

ohmalone

Joined: 10/29/2010

Points: 23

the step "Deploy the KMC uiConf"
running the command "php /opt/kaltura/app/deployment/uiconf/deploy_v2.php –ini=/opt/kaltura/web/flash/kmc/v4.0.12.5/config.ini"
this option i not found the directory "v4.0.12.5" in directory "kmc" and file "config.ini"
in the kmc directory i found the as follow
- account
- analytics
- appstudio
- content
- dashboard
- login
How you running the command?

and setting file 'kConf.php'
i don't same your change
"kmc_rna_version" => 'v1.1.5' -> chris
"kmc_rna_version" => 'v1.1.8.4' -> me
Can you explain please?

Sat, 06/23/2012 - 11:44

#14

Ronileco

Joined: 11/02/2010

Points: 4

Hi guys,
i opened this thread a while ago about migrating Kaltura CE4.0 to CE5.0 http://www.kaltura.org/kaltura-ce-40-50-migration-thread
Hopes it helps: http://www.kaltura.org/kaltura-ce-40-50-migration-thread
i think CE 4.0 to CE5.0 should be discussed in that thread so we can separate the two processes.

Fri, 07/13/2012 - 05:08

#15

jshupe

Joined: 10/13/2011

Points: 2

neuron005, I'm having the same exact issue after upgrading from 4 to 5 that you had in your post dated Tue, 03/06/2012 - 14:39, regarding not being able to upload new videos (they get stuck in queue.)

Have you made any progress in correcting the issue?

Fri, 07/13/2012 - 19:00

#16

neuron005

Joined: 10/05/2011

Points: 103

Hi there
You can follow the same here
http://oodlestechnologies.com/blogs/Upgrading-from-Kaltura-CE4--to-CE5