MogileFS Installation

This installation has been tested with RHEL 5.2 x86_64, the same procedure may work with other distributions as well.

Prerequisites

1. You need aleast two machines to configure MogilesFS, in one machine we have configured the DB, tracker and storage, other machine has only tracker and storage

2. Install elinks and ncftp-client, these are the prerequisite for cpan. In this installation we have used cpan to install the prerequisites.

The elinks packages is being shipped with RHEL by default. If you have installed RHEL with minimal package set you need to install it using rpm or yum.

The ncftp-client can be downloaded from the follwoing location.

http://www.ncftp.com/download/

# tar -jxvf ncftp-3.2.2-src.tar.bz2
# cd ncftp-3.2.2
# ./configure
# make
# make install
# ldconfig

3. Configure cpan, if you are using it for the first time.

# cpan (follow theinstaructios, for first time only)

4. Install the following perl modules.

# cpan install IO::AIO
# cpan install Linux::AIO
# cpan install ExtUtils::MakeMaker
# cpan install Danga::Socket
# cpan install Gearman::Client
# cpan install Gearman::Server
# cpan install Gearman::Client::Async
# cpan install Net::Netmask
# cpan install Perlbal
# cpan install Sys::Syscall
# cpan install MogileFS::Client
# cpan install DBD::mysql

5. Install mogilefs-utils

Do an svn export/checkout to get the latest version of the utils

# svn export http://code.sixapart.com/svn/mogilefs/trunk/utils mogilefs-utils
# perl Makefile.PL
# make
# make test
# make install

Install MogileFS Server

1. Download the latest stable release of MogileFS Server package from cpan web site.

http://cpan.oss.eznetsols.org/authors/id/D/DO/DORMANDO/

# tar -jxvf mogilefs-server-2.20.tar.gz
# cd mogilefs-server-2.20
# perl Makefile.PL
# make
# make test
# make install

2. Install MogileFS APIs, required for the utils

Do an svn export/checkout to get the latest version of the APIs

# svn checkout http://code.sixapart.com/svn/mogilefs/trunk/api/perl/mogilefs-api
# perl Makefile.PL
# make
# make install

NOTE: There are two sub-directories inside this directory and you need to install both of them.

Install MySQL Server

1. MySQL is beeing shipped with RHEL by default, use rpm or yum to install the same. You can also setup multi master replication for better redundancy in the DB level

2. Crate the database for MogileFS

# mysql
mysql> CREATE DATABASE mogilefs;
mysql> GRANT ALL ON mogilefs.* TO 'mogile'@'%';
mysql> SET PASSWORD FOR 'mogile'@'%' = OLD_PASSWORD( 'sekrit' );
mysql> FLUSH PRIVILEGES;
mysql> quit

3. Database Configuration

The database which we have created is empty and will need a schema applied.

# mogdbsetup --dbhost=192.168.32.25 --dbname=mogilefs --dbuser=mogile --dbpassword=sekrit


Configure the tracker

1. On each tracker server create a configuration file at /etc/mogilefs/mogilefsd.conf with the following entries.

db_dsn = DBI:mysql:mogilefs:192.168.32.25
db_user = mogile
db_pass = sekrit
listen = 0.0.0.0:6001
conf_port = 6001
listener_jobs = 10
delete_jobs = 1
replicate_jobs = 5
mog_root = /var/mogdata
reaper_jobs = 1

9. The tracker deamon won't work as root, so create a user for mogilefsd

# adduser mogile

10. Starting Trackers

# su mogile
$ mogilefsd -c /etc/mogilefs/mogilefsd.conf --daemon
$ exit


Storage Node Configuration

1. On each storage server, create the storage directory (make sure it has access permissions for the user you will use to run mogstored):

# mkdir /var/mogdata

2. On each storage server, create a configuration file at /etc/mogilefs/mogstored.conf with the following:

maxconns = 10000
httplisten = 0.0.0.0:7500
mgmtlisten = 0.0.0.0:7501
docroot = /var/mogdata


Configuring mogadm

1. Create a configuration file at /etc/mogilefs/mogilefs.conf with the following

trackers = 192.168.32.25:6001, 192.168.32.26:6001


Configuring mogtool

1. Create a configuration file at /etc/mogilefs/mogtool.conf with the following entries

trackers = 192.168.32.25:6001, 192.168.32.26:6001
domain = testdomain
class = testdata
lib = /usr/lib/perl5/vendor_perl/5.8.8/
overwrite = 1
chunksize = 32M
receipt = admin@bigmaps.com
verify = 1
concurrent = 3


Adding Storage nodes and devices

1. Use the following command to add the storage hosts.

# mogadm host add mogilestorage1 --ip=192.168.32.25 --port=7500 --status=alive
# mogadm host add mogilestorage2 --ip=192.168.32.27 --port=7500 --status=alive

2. Verify this by using following command

# mogadm host list

3. Adding Device for the Storage host

# mogadm device add mogilestorage1 1
# mogadm device add mogilestorage2 2

NOTE: Device ID should be unique, verify this by using following command

# mogadm device list

4. Create Device directory on all Storage Servers.

# mkdir -p /var/mogdata/dev1
# mkdir -p /var/mogdata/dev1

5. Start Storage Server

mogstored --daemon


Testing

1. Create a domain:

# mogadm domain add testdomain

2. Add a class to the domain:

# mogadm class add testdomain testclass

3. Use modtool for adding and retriving the data.

# mogtool inject file-name key-name
# mogtool extract key-name file-name

for large files, >64M

# mogtool inject --bigfile file-name key-name
# mogtool extract --bigfile key-name file-name

for directories

# mogtool inject --bigfile dir-name key-name
# mogtool extract --bigfile dir-name file-name