Fli
08-17-2018, 04:49 PM
We have a dedicated server which is hosting a VPS with WHM/cPanel installed and many cPanel accounts
We need to setup new cpanel server (or restore old cpanel server VPS backup so it can be used as a new server), to which we will move all cpanel accounts or cpanel full backups with minimum downtime.
(in my case problem is that i wanted to downgrade WHM, but unable to do so so i decided to restore my OpenVZ VPS backup and migrate accounts with minimum downtime.
This tutorial is not complete (and not fully tested, use at your own risk), but shows possible way to handle the process
This tutorial can work for any server migration probably or for any virtualization type probably, but in my case i am on an OpenVZ virtualization dedicated server.
SUMARIZED theory of migration
- delete all accounts on 880 - for cpanel in $(ls -A1 /var/cpanel/users|grep -v system);do /scripts/removeacct "$cpanel";done
- spot sites/mysqls frequently updated and write down database names
- setup backup process to noon so backups are completed when i am available to do the work
- restore backups on new vps (Note: /scripts/restorepkg restoring suspended accounts as not suspended!)
- do io sensitive files rsyncing on background?
- use mysql db names of frequently updated sites in exporting bash script (script is below) and then importing at the end, i should terminate rsyncing of the files and remove server Ip from old server and add it to new one and checking if things works good
- service start crond (on new server)
- (this step only valid for me) enable backing up of the new vps in monthly backup bash script
- (this step only valid for me) uncomment two backup lines from crontab:
#0 1 * * * /usr/local/cpanel/scripts/cpbackup
#0 2 * * * /usr/local/cpanel/bin/backup
DETAILED theory of migration
1. I restore my VPS backup as a secondary VPS so my live and restored VPS run both in paralel. Nice & ionice to prefer other important processes over restoration task:
# yum install screen -y -q;screen
# /bin/nice -n 19 /usr/bin/ionice -c2 -n7 vzdump --restore vzdump-870.tgz 880
After about hour, it finished:
INFO: extracting archive 'vzdump-870.tgz'
INFO: Total bytes read: 62262169600 (58GiB, 15MiB/s)
INFO: extracting configuration to '/etc/vz/conf/880.conf'
INFO: restore successful
so i exit the"screen" session:
# exit
I deleted all IPs from the newly restored VPS and lowered number of CPUs & RAM on it, and started VPS:
# vzctl set 880 --ipdel all --save
# vzctl set 880 --numproc 7 --save
# vzctl set 880 --ram 4G --save
# vzctl --verbose start 880
After start i disabled UPCP as i have not wanted server be updated:
# mv /scripts/upcp /scripts/upcp-do-not-update
# crontab -e # comment out line containing upcp
In case new server that we preparing contains cpanel accounts (is restored from backup), then Delete all cpanels (on new server that is not live) so we can then restore there all fullbackups that will be made on live server. Either wait for WHM daily backup to complete them or do manually:
# for cpanel in $(ls -A1 /var/cpanel/users|grep -v system);do /scripts/pkgacct "$cpanel";done
Once cpanel accounts backups are completed, we need to copy them to new server and restore them from within new server (probably "screen" session):
Something like (untested):
cd /path/to/cpanel-backups/location
for backupfile in $(ls -A1 ./|grep "tar.gz");do /scripts/restorepkg --allow_reseller "$backupfile";done;echo "cPanel full backups restore should be complete"|mail -s "cPanels restore finished" -- [email protected]
Once backup is done, we can execute infinite rsync command, like:
while true;do rsync -aqHl --exclude=virtfs/ --delete options -e ssh /home/* root@destinationip:/home/;done
(this one is untested, you should first verify rsync parameters. The path are set the way command be executed on the old server. The commend can be executed in separate terminal session or via "screen" Linux utility.)
Once rsync ran at least once and /home data seems quite similar, it an be the time to do MySQL bakcups and restoring and them immediately after that switchin server IPs directing the traffic to new server:
#!/bin/bash
#set -ex
# This script should be run as a root on a WHM/cPanel server. Script will:
# backup users MySQL databases (located in default /var/lib/mysql location)
cpusers="$(ls -A1 /var/cpanel/users)"
cpanelusernameswhichmysqlstoexclude="excludedcpanelusername|anotherusername|evenpartial"
mysqldatabases="$(ls -A1 /var/lib/mysql|grep _|grep -Ev ".pid|.err|schema|ib_logfile|RPM_UPGRADE|upgrade_in fo|$cpanelusernameswhichmysqlstoexclude")"
#echo -e "Source cpanels\n$cpusers\n"
#echo -e "Source DBs:\n$mysqldatabases\n"
#echo -e "Excluded:\n$cpanelusernameswhichmysqlstoexclude\n"
rm -f mysqls_output_tmp mysqldatabases_output_real
echo "$mysqldatabases" > mysqls_output_tmp
# create list of only realy databases of cpanel users
for user in $(echo "$cpusers");do
if [[ "$(grep "$user" mysqls_output_tmp)" == *"$user"* ]];then grep "$user" mysqls_output_tmp|grep "_" >> mysqldatabases_output_real;fi
done
# show db list and prompt
cat mysqldatabases_output_real
echo -e "Resulting filtered list of MySQL databases to backup. Proceed with backup? (y to proceed, other key to exit)"
read input;if [[ "$input" != "y" ]];then exit;fi
# backup the databases now
for dbname in $(cat mysqldatabases_output_real);do
/bin/nice -n 19 /usr/bin/ionice -c2 -n7 mysqldump -u root "$dbname" > "$dbname".sql
# stat "$dbname".sql
# echo "hit any key to continue with next DB or Ctrl plus C to abort"
# read input
done
# make the DB read only on source (i would do it because i am using this script in migration, i will then switch Ip from old to new server)
# The command to make the DB read-only is missing so far in this script
rm -f mysqls_output_tmp mysqldatabases_output_real
Above script is missing the command to move backups to new server.
Following restoration script (executed on new server in the directory where we moved mysql backups) should restore them and replace any existing ones.
#!/bin/bash
#set -ex
# restore the cpanel MySQL databases, script should be placed and executed from directory which contains all .sql files (cpanelusername_dbname.sql)
cpusers="$(ls -A1 /var/cpanel/users)"
cpanelusernameswhichmysqlstoexclude="excludedcpanelusername|anotherusername|evenpartial"
mysqldatabases="$(ls -A1 /var/lib/mysql|grep _|grep -Ev ".pid|.err|schema|ib_logfile|RPM_UPGRADE|upgrade_in fo|$cpanelusernameswhichmysqlstoexclude")"
#echo -e "Source cpanels\n$cpusers\n"
#echo -e "Source DBs:\n$mysqldatabases\n"
#echo -e "Excluded:\n$cpanelusernameswhichmysqlstoexclude\n"
rm -f mysqls_output_tmp mysqldatabases_output_real
echo "$mysqldatabases" > mysqls_output_tmp
# create list of only realy databases of cpanel users
for user in $(echo "$cpusers");do
if [[ "$(grep "$user" mysqls_output_tmp)" == *"$user"* ]];then grep "$user" mysqls_output_tmp|grep "_" >> mysqldatabases_output_real;fi
done
for dbname in $(cat mysqldatabases_output_real);do
/bin/nice -n 19 /usr/bin/ionice -c2 -n7 mysql -u root "$dbname" < "$dbname".sql
done
rm -f mysqls_output_tmp mysqldatabases_output_real
echo "MySQL restore completed"|mail -s "MySQL Restore finished, switch server IP to prevent loosing MySQL data because visitors contributing to old server" -- [email protected]
# Here a missing command (maybe ssh) to instruct switching of the IPs
#
above script is at the end missing command to switch VPS IPs. Script is not tested.
We need to setup new cpanel server (or restore old cpanel server VPS backup so it can be used as a new server), to which we will move all cpanel accounts or cpanel full backups with minimum downtime.
(in my case problem is that i wanted to downgrade WHM, but unable to do so so i decided to restore my OpenVZ VPS backup and migrate accounts with minimum downtime.
This tutorial is not complete (and not fully tested, use at your own risk), but shows possible way to handle the process
This tutorial can work for any server migration probably or for any virtualization type probably, but in my case i am on an OpenVZ virtualization dedicated server.
SUMARIZED theory of migration
- delete all accounts on 880 - for cpanel in $(ls -A1 /var/cpanel/users|grep -v system);do /scripts/removeacct "$cpanel";done
- spot sites/mysqls frequently updated and write down database names
- setup backup process to noon so backups are completed when i am available to do the work
- restore backups on new vps (Note: /scripts/restorepkg restoring suspended accounts as not suspended!)
- do io sensitive files rsyncing on background?
- use mysql db names of frequently updated sites in exporting bash script (script is below) and then importing at the end, i should terminate rsyncing of the files and remove server Ip from old server and add it to new one and checking if things works good
- service start crond (on new server)
- (this step only valid for me) enable backing up of the new vps in monthly backup bash script
- (this step only valid for me) uncomment two backup lines from crontab:
#0 1 * * * /usr/local/cpanel/scripts/cpbackup
#0 2 * * * /usr/local/cpanel/bin/backup
DETAILED theory of migration
1. I restore my VPS backup as a secondary VPS so my live and restored VPS run both in paralel. Nice & ionice to prefer other important processes over restoration task:
# yum install screen -y -q;screen
# /bin/nice -n 19 /usr/bin/ionice -c2 -n7 vzdump --restore vzdump-870.tgz 880
After about hour, it finished:
INFO: extracting archive 'vzdump-870.tgz'
INFO: Total bytes read: 62262169600 (58GiB, 15MiB/s)
INFO: extracting configuration to '/etc/vz/conf/880.conf'
INFO: restore successful
so i exit the"screen" session:
# exit
I deleted all IPs from the newly restored VPS and lowered number of CPUs & RAM on it, and started VPS:
# vzctl set 880 --ipdel all --save
# vzctl set 880 --numproc 7 --save
# vzctl set 880 --ram 4G --save
# vzctl --verbose start 880
After start i disabled UPCP as i have not wanted server be updated:
# mv /scripts/upcp /scripts/upcp-do-not-update
# crontab -e # comment out line containing upcp
In case new server that we preparing contains cpanel accounts (is restored from backup), then Delete all cpanels (on new server that is not live) so we can then restore there all fullbackups that will be made on live server. Either wait for WHM daily backup to complete them or do manually:
# for cpanel in $(ls -A1 /var/cpanel/users|grep -v system);do /scripts/pkgacct "$cpanel";done
Once cpanel accounts backups are completed, we need to copy them to new server and restore them from within new server (probably "screen" session):
Something like (untested):
cd /path/to/cpanel-backups/location
for backupfile in $(ls -A1 ./|grep "tar.gz");do /scripts/restorepkg --allow_reseller "$backupfile";done;echo "cPanel full backups restore should be complete"|mail -s "cPanels restore finished" -- [email protected]
Once backup is done, we can execute infinite rsync command, like:
while true;do rsync -aqHl --exclude=virtfs/ --delete options -e ssh /home/* root@destinationip:/home/;done
(this one is untested, you should first verify rsync parameters. The path are set the way command be executed on the old server. The commend can be executed in separate terminal session or via "screen" Linux utility.)
Once rsync ran at least once and /home data seems quite similar, it an be the time to do MySQL bakcups and restoring and them immediately after that switchin server IPs directing the traffic to new server:
#!/bin/bash
#set -ex
# This script should be run as a root on a WHM/cPanel server. Script will:
# backup users MySQL databases (located in default /var/lib/mysql location)
cpusers="$(ls -A1 /var/cpanel/users)"
cpanelusernameswhichmysqlstoexclude="excludedcpanelusername|anotherusername|evenpartial"
mysqldatabases="$(ls -A1 /var/lib/mysql|grep _|grep -Ev ".pid|.err|schema|ib_logfile|RPM_UPGRADE|upgrade_in fo|$cpanelusernameswhichmysqlstoexclude")"
#echo -e "Source cpanels\n$cpusers\n"
#echo -e "Source DBs:\n$mysqldatabases\n"
#echo -e "Excluded:\n$cpanelusernameswhichmysqlstoexclude\n"
rm -f mysqls_output_tmp mysqldatabases_output_real
echo "$mysqldatabases" > mysqls_output_tmp
# create list of only realy databases of cpanel users
for user in $(echo "$cpusers");do
if [[ "$(grep "$user" mysqls_output_tmp)" == *"$user"* ]];then grep "$user" mysqls_output_tmp|grep "_" >> mysqldatabases_output_real;fi
done
# show db list and prompt
cat mysqldatabases_output_real
echo -e "Resulting filtered list of MySQL databases to backup. Proceed with backup? (y to proceed, other key to exit)"
read input;if [[ "$input" != "y" ]];then exit;fi
# backup the databases now
for dbname in $(cat mysqldatabases_output_real);do
/bin/nice -n 19 /usr/bin/ionice -c2 -n7 mysqldump -u root "$dbname" > "$dbname".sql
# stat "$dbname".sql
# echo "hit any key to continue with next DB or Ctrl plus C to abort"
# read input
done
# make the DB read only on source (i would do it because i am using this script in migration, i will then switch Ip from old to new server)
# The command to make the DB read-only is missing so far in this script
rm -f mysqls_output_tmp mysqldatabases_output_real
Above script is missing the command to move backups to new server.
Following restoration script (executed on new server in the directory where we moved mysql backups) should restore them and replace any existing ones.
#!/bin/bash
#set -ex
# restore the cpanel MySQL databases, script should be placed and executed from directory which contains all .sql files (cpanelusername_dbname.sql)
cpusers="$(ls -A1 /var/cpanel/users)"
cpanelusernameswhichmysqlstoexclude="excludedcpanelusername|anotherusername|evenpartial"
mysqldatabases="$(ls -A1 /var/lib/mysql|grep _|grep -Ev ".pid|.err|schema|ib_logfile|RPM_UPGRADE|upgrade_in fo|$cpanelusernameswhichmysqlstoexclude")"
#echo -e "Source cpanels\n$cpusers\n"
#echo -e "Source DBs:\n$mysqldatabases\n"
#echo -e "Excluded:\n$cpanelusernameswhichmysqlstoexclude\n"
rm -f mysqls_output_tmp mysqldatabases_output_real
echo "$mysqldatabases" > mysqls_output_tmp
# create list of only realy databases of cpanel users
for user in $(echo "$cpusers");do
if [[ "$(grep "$user" mysqls_output_tmp)" == *"$user"* ]];then grep "$user" mysqls_output_tmp|grep "_" >> mysqldatabases_output_real;fi
done
for dbname in $(cat mysqldatabases_output_real);do
/bin/nice -n 19 /usr/bin/ionice -c2 -n7 mysql -u root "$dbname" < "$dbname".sql
done
rm -f mysqls_output_tmp mysqldatabases_output_real
echo "MySQL restore completed"|mail -s "MySQL Restore finished, switch server IP to prevent loosing MySQL data because visitors contributing to old server" -- [email protected]
# Here a missing command (maybe ssh) to instruct switching of the IPs
#
above script is at the end missing command to switch VPS IPs. Script is not tested.