Fli
04-04-2013, 02:06 AM
It is very simple to do it. But still i dont guarantee it will work. Try this guide on your own risk. Please note that HyperVM works only on Cent OS 5.x, it did not worked on 6.x. Also it was tested on 64bit system, but 32bit should work too. If anyone need to use CentOS 6.x, one can install HyperVM Development version (http://internetlifeforum.com/virtualisation/2494-hypervm-development-version-installation/) which is not recommended for production servers.
Hardware for openvz server:
for VPS reselling, one should not use single HDD server, instead one may use:
- two same HDDs in raid 1 (one drive can fail and server continue working)
- three HDDs in raid 1 (more reading speed and 2 drives can fail)
- four HDDs in raid 10 - (4x read, 2x write, 1-2 drives can fail)
- one SSD for operating system & VPSes and second drive for frequent backups, good idea to clone SSD(or HDD) to HDD (http://internetlifeforum.com/linux-forums/2485-how-clone-hdd-remote-server-via-ipmi-drac-kvm-ip/) after initial OS and hyperVM install. Then when SSD fail, one can reset server, login IPMI/DRAC console and at boot go to bios by "Del" key, then change HDD boot priority to boot from backup drive which was cloned from currently failed drive. Then restore just VMs backups we did.
------------
OpenVZ/HyperVM installation
here is the manual: http://lxcenter.org/software/hypervm/hypervm-installation
following manual is made out of above tutorial plus added another missing commands.
Here is bash script for automated installation! (it can replace tutorial on this page)> http://internetlifeforum.com/virtualisation/2607-script-install-openvz-hypervm-epel-vzdump-centos/
The commands to do before installation:
su -
setenforce 0
cat /etc/sysconfig/selinux | grep SELINUX=
selinux must be disabled above
Select --virtualization-type bellow, its highlighted in bold. If you have installed one already, select type=NONE, if you want hypervm to install OpenVZ, then =openvz
wget http://download.hypervm-ng.org/download/legacy/production/hypervm-install-master.sh
sh ./hypervm-install-master.sh --virtualization-type=xen/openvz/NONE
Update linux software (was needed to sendmail to work;it can be ~70MB data), install, enable sendmail and restart it
yum update -y;yum install sendmail* -y;chkconfig sendmail on;service sendmail restart
restart server
chkconfig iptables on;reboot
check grub boot loader to see if there is openvz group and default=0
cat /etc/grub.conf
above command shows "default=0" and openvz part is on the first position, like below: (it is as it should be)
default=0
timeout=5
title OpenVZ (2.6.18-398.el5.028stab116.1)
kernel /boot/vmlinuz-2.6.18-398.el5.028stab116.1 root=/dev/sda1 ro selinux=0
root (hd0,0)
initrd /boot/initrd-2.6.18-398.el5.028stab116.1.img
title linux centos5_64
kernel /boot/bzImage-3.10.23-xxxx-std-ipv6-64 root=/dev/sda1 ro
root (hd0,0)
IF OpenVZ kernel is not shown by that command, then you need to:
After installation, If you selected openvz or xen, you will need to edit:
vi /etc/grub.conf
and
vi /boot/grub/grub.conf
usually you will have there around 3 options in above files: Title CentOS, Title OpenVZ, Title CentOS
the first "title" is 0, second 1, third 2. So in this case i select 1 (openvz): => default=1
(OpenVZ must be selected if you picked virtualization openvz on hypervm install above)
if you changed anything in above grub files, you need to do command: reboot
if you wrongly modiffied grub, computer may not start
endIF
login hypervm
http://yourserverip:8888
password: admin
Opening hypervm ports may be needed, im not sure, but when connection issue, one may try
iptables -I INPUT -p tcp --dport 8888 -j ACCEPT;iptables -I INPUT -p udp --dport 8888 -j ACCEPT;iptables -I INPUT -p tcp --dport 8887 -j ACCEPT;iptables -I INPUT -p udp --dport 8887 -j ACCEPT;iptables -I INPUT -p tcp --dport 8889 -j ACCEPT;iptables -I INPUT -p udp --dport 8889 -j ACCEPT
8888 hypervm web panel
8887 https hypervm web panel
8889 connections between 2+ hypervm servers
change password, enable guard, and on same page click Whitelist tab to whitelist your IP/s
You may receive this error then after login into HyperVM:
"hyperVM could not connect to an smtp server on this server."
do this via SSH/CLI: chkconfig sendmail on;service sendmail restart
Next important things to do after HyperVM,OpenVZ install:
- Install vzdump (http://internetlifeforum.com/virtualisation/2495-how-install-openvz-vzdump/)
- Move /vz folder to the partition with enough disk space (http://internetlifeforum.com/virtualisation/250-how-free-up-vz-space-if-its-full-how-move/)
- Install EPEL repo (http://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/), then yum install sysstat ntp iftop iotop iostat vmstat htop mutt;
- setup UTC time (http://internetlifeforum.com/linux-forums/2606-how-setup-utc-time-time-synchronization-linux-quickly/)
- Schedule regular automatic backups of HyperVM mysql database (http://internetlifeforum.com/virtualisation/1150-how-backup-restore-hypervm-settings-config-database/)
- Allow creating more than 10 VPSs (http://internetlifeforum.com/virtualisation/2435-hypervm-error-alert-could_not_connect_to_license_server/)
- Install defending tools 1) DDoS deflate (http://internetlifeforum.com/security-protection/1132-how-manage-ddos-deflate-deny-ddos-attack/#post1675), 2) Fail2Ban (http://internetlifeforum.com/security-protection/1136-installing-fail2ban-linux-prevent-various-bad-bots-attacks-bruteforce/), 3) NodeWatch (http://internetlifeforum.com/virtualisation/2442-nodewatch-installation/)
- check OpenVZ script to periodically check & suspend/restart overloaded VMs (VPSs) (http://internetlifeforum.com/virtualisation/2313-openvz-script-periodically-check-suspend-restart-overloaded-vms-vpss/)
- Change SSH port from 22 to some 2-5 digits number that is not used by other service (edit /etc/ssh/sshd_config and restart ssh service - service ssh restart)
- Setup automatic regular backup of all OpenVZ VMs (http://internetlifeforum.com/virtualisation/716-how-backup-restore-all-hypervm-openvz-vms-virtual-machines-vpss/) and then setup offserver backup (example to some cheap Kimsufi server) and prepare tutorial on how to quickly restore backup if primary server is not accessible or suspended
- check that in /etc/vz/conf/ are files like "ve-vswap-256m.conf-sample" which contains all needed VPS parameters. My example file is:
# RAM
PHYSPAGES="0:256M"
# Swap
SWAPPAGES="0:512M"
# Disk quota parameters (in form of softlimit:hardlimit)
DISKSPACE="2G:2.2G"
DISKINODES="131072:144179"
QUOTATIME="0"
# CPU fair scheduler parameter
CPUUNITS="1000"
NETFILTER="full"
# i added these manually
DEVNODES="net/tun:rw"
DEVICES=""
CAPABILITY="NET_ADMIN:on"
# IOLIMIT 5MB rw per second
IOLIMIT="5242880"
IOPSLIMIT="30"
When PHYSPAGES used like above and PRIVVMPAGES not used (or unlimited?), it should enable vSWAP which seems to behave significantly better.
- One may also try to use "ipset" (https://internetlifeforum.com/security-protection/8900-tutorial-how-install-use-ipset/) to block large number of bad IPs before they reach VPSs. ipset rules would have to be added probably not into input chain but into forward or prerouting.
Hardware for openvz server:
for VPS reselling, one should not use single HDD server, instead one may use:
- two same HDDs in raid 1 (one drive can fail and server continue working)
- three HDDs in raid 1 (more reading speed and 2 drives can fail)
- four HDDs in raid 10 - (4x read, 2x write, 1-2 drives can fail)
- one SSD for operating system & VPSes and second drive for frequent backups, good idea to clone SSD(or HDD) to HDD (http://internetlifeforum.com/linux-forums/2485-how-clone-hdd-remote-server-via-ipmi-drac-kvm-ip/) after initial OS and hyperVM install. Then when SSD fail, one can reset server, login IPMI/DRAC console and at boot go to bios by "Del" key, then change HDD boot priority to boot from backup drive which was cloned from currently failed drive. Then restore just VMs backups we did.
------------
OpenVZ/HyperVM installation
here is the manual: http://lxcenter.org/software/hypervm/hypervm-installation
following manual is made out of above tutorial plus added another missing commands.
Here is bash script for automated installation! (it can replace tutorial on this page)> http://internetlifeforum.com/virtualisation/2607-script-install-openvz-hypervm-epel-vzdump-centos/
The commands to do before installation:
su -
setenforce 0
cat /etc/sysconfig/selinux | grep SELINUX=
selinux must be disabled above
Select --virtualization-type bellow, its highlighted in bold. If you have installed one already, select type=NONE, if you want hypervm to install OpenVZ, then =openvz
wget http://download.hypervm-ng.org/download/legacy/production/hypervm-install-master.sh
sh ./hypervm-install-master.sh --virtualization-type=xen/openvz/NONE
Update linux software (was needed to sendmail to work;it can be ~70MB data), install, enable sendmail and restart it
yum update -y;yum install sendmail* -y;chkconfig sendmail on;service sendmail restart
restart server
chkconfig iptables on;reboot
check grub boot loader to see if there is openvz group and default=0
cat /etc/grub.conf
above command shows "default=0" and openvz part is on the first position, like below: (it is as it should be)
default=0
timeout=5
title OpenVZ (2.6.18-398.el5.028stab116.1)
kernel /boot/vmlinuz-2.6.18-398.el5.028stab116.1 root=/dev/sda1 ro selinux=0
root (hd0,0)
initrd /boot/initrd-2.6.18-398.el5.028stab116.1.img
title linux centos5_64
kernel /boot/bzImage-3.10.23-xxxx-std-ipv6-64 root=/dev/sda1 ro
root (hd0,0)
IF OpenVZ kernel is not shown by that command, then you need to:
After installation, If you selected openvz or xen, you will need to edit:
vi /etc/grub.conf
and
vi /boot/grub/grub.conf
usually you will have there around 3 options in above files: Title CentOS, Title OpenVZ, Title CentOS
the first "title" is 0, second 1, third 2. So in this case i select 1 (openvz): => default=1
(OpenVZ must be selected if you picked virtualization openvz on hypervm install above)
if you changed anything in above grub files, you need to do command: reboot
if you wrongly modiffied grub, computer may not start
endIF
login hypervm
http://yourserverip:8888
password: admin
Opening hypervm ports may be needed, im not sure, but when connection issue, one may try
iptables -I INPUT -p tcp --dport 8888 -j ACCEPT;iptables -I INPUT -p udp --dport 8888 -j ACCEPT;iptables -I INPUT -p tcp --dport 8887 -j ACCEPT;iptables -I INPUT -p udp --dport 8887 -j ACCEPT;iptables -I INPUT -p tcp --dport 8889 -j ACCEPT;iptables -I INPUT -p udp --dport 8889 -j ACCEPT
8888 hypervm web panel
8887 https hypervm web panel
8889 connections between 2+ hypervm servers
change password, enable guard, and on same page click Whitelist tab to whitelist your IP/s
You may receive this error then after login into HyperVM:
"hyperVM could not connect to an smtp server on this server."
do this via SSH/CLI: chkconfig sendmail on;service sendmail restart
Next important things to do after HyperVM,OpenVZ install:
- Install vzdump (http://internetlifeforum.com/virtualisation/2495-how-install-openvz-vzdump/)
- Move /vz folder to the partition with enough disk space (http://internetlifeforum.com/virtualisation/250-how-free-up-vz-space-if-its-full-how-move/)
- Install EPEL repo (http://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/), then yum install sysstat ntp iftop iotop iostat vmstat htop mutt;
- setup UTC time (http://internetlifeforum.com/linux-forums/2606-how-setup-utc-time-time-synchronization-linux-quickly/)
- Schedule regular automatic backups of HyperVM mysql database (http://internetlifeforum.com/virtualisation/1150-how-backup-restore-hypervm-settings-config-database/)
- Allow creating more than 10 VPSs (http://internetlifeforum.com/virtualisation/2435-hypervm-error-alert-could_not_connect_to_license_server/)
- Install defending tools 1) DDoS deflate (http://internetlifeforum.com/security-protection/1132-how-manage-ddos-deflate-deny-ddos-attack/#post1675), 2) Fail2Ban (http://internetlifeforum.com/security-protection/1136-installing-fail2ban-linux-prevent-various-bad-bots-attacks-bruteforce/), 3) NodeWatch (http://internetlifeforum.com/virtualisation/2442-nodewatch-installation/)
- check OpenVZ script to periodically check & suspend/restart overloaded VMs (VPSs) (http://internetlifeforum.com/virtualisation/2313-openvz-script-periodically-check-suspend-restart-overloaded-vms-vpss/)
- Change SSH port from 22 to some 2-5 digits number that is not used by other service (edit /etc/ssh/sshd_config and restart ssh service - service ssh restart)
- Setup automatic regular backup of all OpenVZ VMs (http://internetlifeforum.com/virtualisation/716-how-backup-restore-all-hypervm-openvz-vms-virtual-machines-vpss/) and then setup offserver backup (example to some cheap Kimsufi server) and prepare tutorial on how to quickly restore backup if primary server is not accessible or suspended
- check that in /etc/vz/conf/ are files like "ve-vswap-256m.conf-sample" which contains all needed VPS parameters. My example file is:
# RAM
PHYSPAGES="0:256M"
# Swap
SWAPPAGES="0:512M"
# Disk quota parameters (in form of softlimit:hardlimit)
DISKSPACE="2G:2.2G"
DISKINODES="131072:144179"
QUOTATIME="0"
# CPU fair scheduler parameter
CPUUNITS="1000"
NETFILTER="full"
# i added these manually
DEVNODES="net/tun:rw"
DEVICES=""
CAPABILITY="NET_ADMIN:on"
# IOLIMIT 5MB rw per second
IOLIMIT="5242880"
IOPSLIMIT="30"
When PHYSPAGES used like above and PRIVVMPAGES not used (or unlimited?), it should enable vSWAP which seems to behave significantly better.
- One may also try to use "ipset" (https://internetlifeforum.com/security-protection/8900-tutorial-how-install-use-ipset/) to block large number of bad IPs before they reach VPSs. ipset rules would have to be added probably not into input chain but into forward or prerouting.