Copyright © 2012 Kaltura Inc.
All Rights Reserved. Designated trademarks and brands are the property of their respective owners.
Use of this web site constitutes acceptance of the Terms of Use and Privacy Policy.
EduVideo.org
Which Kaltura working directories have high storage needs (mySql, apache, both)?
Mon, 07/04/2011 - 18:30
Hi
I have a very quick / easy question (hopefully), which I couldn't find the answer too in this forum / documentation.
Here;'s the background....I'm in the process of building a new Kaltura 4.0 CE server (using Amazon Web Services / EC2 / EBS) on a Linux distro. We have our initial capacity planning requires an Amazon EBS volume size of 500GB, which we'll extend across more EBS volumes as capacity demands/dictates. I've yet to evaluate this, but we're currently assuming we can use Logical Volume Manager (LVM) to facilitate growth over multiple EBS volumes.
My question...I'd like to mount the new EBS volume and ensure that all the video data that Kaltura stores and streams is on the EBS volume. Is it safe to assume that the only folders I need to concern myself with the apache www (i.e.
/var/www/) and the data files used by MySQL (i.e. /var/lib/mysql/*.)
is having both these folders on the Amazon EBS correct - or do i just need one of the two (i.e. mySQL)? Like i said, I've scanned the documentation but it didn't jump out at me - so apologies if this is a noob question.
All advice/recommendations welcome :)
Gavin.
Tue, 07/05/2011 - 15:14
#2
Hi papyromancer
Thanks for the details, I have incorporated that into my AWS EC2 installation configuration. Here's how it looks:
1. Amazon Machine Instance (AMI)
"Large" Instance, running Ubuntu 10.4 x64 (ami-ce38c3a7), using 8GB EBS.
NOTE: Internal storage that comes with "Large" is not being used. We felt that snap shot benefits of a pure persistent EBS storage was worth the cost.
2. Amazon Block Storage (EBS): 500GB, attached to AMI and managed as a Logical Volume (LVM).
With regards to the storage I have mounted that as /aws-ebs/ and underneath that folder I have the following:
/aws-ebs/opt/kaltura <--- "Kaltura installation directory"
/aws-ebs/usr/local/pentaho <--- "Panteho Installation directory"
/aws-ebs/var/www <--- Default apache root folder
/aws-ebs/var/lib/mysql <--- "MySQL data file folder"
Kaltura & Pentaho were native installs into these folders, MySQL and Apache folders were moved. I placed symbolic links in the original locations to mitigate an hard references. If I'm still missing some folders that expand rapidly please feel free to post your thoughts.
FYI - I'm no Linux wiz - so I'm sure there may have been more optimal methods to achieve the above. Any suggestions would and/or to improve mine/community knowledge.
Gavin.
Thu, 09/08/2011 - 08:15
#3
Hello Gavin ,
Did it worked ?
Can your solution scale with 2tb of storage ?
I think most of your website data will be stored in /opt/kaltura and the mysql location will depend on your distribution.
I've been giving scaling a CE on ebs a bit of thought lately:
1. keep mysql and data storage on separate EBS (raided to increase throughput) to make snapshotting backups easier for different schedules
2. keep logs on instance storage
I've had some success with snapshotting a running ce to larger and smaller VM sizes (I'm running 64bit micro and large), but I've had no server load during the transition/elastic ip redirection.
that's just my thoughts, I know of no best practice guide, and I'm no ec2 wizard. I'd love to hear what others are doing.