Nginx proxy https

 

Nginx Self-sign https : http://blog.crois.net/2019/09/23/centos7-nginx-self-signed-https/ 설정하였던 VM 을

이용하여 nginx proxy 를 테스트 합니다. 상단 nginx-proxy 에만 인증서가 있으면 되며 백단 서버 두대의 경우

별도의 인증서 및 설정이 필요 하지 않습니다.

nginx 설치및 php 설치의 경우 이전 포스트를 참고해 주세요.

Nginx-Proxy 서버에서 ssl 설정을 하고 Nginx-www1 / Nginx-www2 에서는 80 port 설정만 진행 합니다.

1.Nginx-proxy 설정

[root@test ~]# vi /etc/nginx/sites-enabled/test_com.conf
server {
    listen       80;
    server_name  www.test.com test.com;
    root   /var/www/html/test.com/public_html;
    index  index.php index.html index.htm;
    location / {
        return 301 https://test.com$request_uri;
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
    }
}

server {
    listen       443 http2 ssl;
    server_name  www.test.com test.com;
    root   /var/www/html/test.com/public_html;
    index  index.php index.html index.htm;

    access_log /var/www/html/test.com/logs/access.log;
    error_log  /var/www/html/test.com/logs/error.log warn;


    ssl_certificate /etc/ssl/certs/nginx-selfsign.crt;
    ssl_certificate_key /etc/ssl/private/nginx-selfsign.key;
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    ssl_session_timeout 10m;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;
    ssl_protocols TLSv1.2;
    ssl_ecdh_curve secp384r1;
    #ssl_ciphers  ECDH+AESGCM:!AES128:!RSA+AES:!aNULL:!MD5:!DSS:!DHE:!kEDH:HIGH:!eNULL:!EXPORT:!DES:!RC4:!PSK:!AECDH:!LOW:!SRP:!ADH:!RSA:TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-256-GCM-SHA384:!COMPLEMENTOFDEFAULT;
    ssl_ciphers AES256+EECDH:AES256+EDH:!aNULL;
    ssl_prefer_server_ciphers on;
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";
#    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;

    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;


    fastcgi_buffering               on;
    fastcgi_buffer_size             16k;
    fastcgi_buffers                 16 16k;

    # time-out settings
    fastcgi_connect_timeout         600s;
    fastcgi_send_timeout            600s;
    fastcgi_read_timeout            600s;

    # php performance settings
    sendfile                        on;
    tcp_nopush                      off;
    keepalive_requests              0;


    location / {
        rewrite ^/(/.*)$ $1 break;
        proxy_pass http://test.com;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_redirect off;
        try_files $uri $uri/ /index.php?$query_string;
    }
}
upstream test.com {
    server 10.10.10.93:80;
    server 10.10.10.94:80;
}
[root@test ~]# systemctl restart nginx

 

 

2. server 설정

[root@www1 ~]# vi /etc/nginx/sites-enabled/test_com.conf
server {
    listen       80;
    server_name  www.test.com test.com;
    root   /var/www/html/test.com/public_html;
    index  index.php index.html index.htm;
    location / {
        try_files $uri $uri/ /index.php?$query_string;
        autoindex on;
    }

    access_log  /var/www/html/test.com/logs/access.log;
    error_log  /var/www/html/test.com/logs/error.log warn;

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

[root@www1 ~]# systemctl restart nginx ; systemctl restart php-fpm

 

 

3. web-site 확인

 

4. server log 확인

tail -f access.log 를 확인 합니다.

Nginx Self-signed https

 

nginx https 설정을 테스트 합니다.

nginx web-server 와 php71 까지 설치후 Test 도메인으로 접속을 테스트 합니다.

 

 

1. Nginx 설치

[root@test ~]# vi /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1
[root@test ~]# yum clean all

[root@test ~]# yum install -y nginx
[root@test ~]# systemctl enable nginx ; systemctl start nginx

 

 

2. php71 설치

[root@test ~]# yum install -y epel-release yum-utils
[root@test ~]# rpm -Uvh http://ftp.riken.jp/Linux/remi/enterprise/remi-release-7.rpm
[root@test ~]# yum-config-manager --enable remi-php71
[root@test ~]# yum -y install php php-mysql php-fpm php-opcache php-gd php-ldap php-odbc php-pear php-xml php-xmlrpc php-mbstring php-soap curl curl-devel
[root@test ~]# vi /etc/php.ini
date.timezone = Asia/Seoul
[root@test ~]# vi /etc/php-fpm.d/www.conf

user = nginx
group = nginx

listen.owner = nginx
listen.group = nginx
listen.mode = 0660

[root@test ~]# systemctl restart php-fpm
[root@test ~]# systemctl enable php-fpm

 

 

3. Nginx 설정

[root@test ~]# mkdir /etc/nginx/sites-enabled
[root@test ~]# vi /etc/nginx/nginx.conf
user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*.conf;
}

[root@test ~]# vi /etc/nginx/conf.d/default.conf
server {
    listen       80 default_server;
    server_name  localhost;

    charset UTF-8;

        root   /usr/share/nginx/html;
        index  index.php index.html index.htm;
    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        #fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

}

[root@test ~]# mkdir -p /var/www/html/test.com/{public_html,logs}
[root@test ~]# vi /etc/nginx/sites-enabled/test_com.conf
server {
    listen       80;
    server_name  www.test.com test.com;
    root   /var/www/html/test.com/public_html;
    index  index.php index.html index.htm;
    location / {
        try_files $uri $uri/ /index.php?$query_string;
        autoindex on;
    }

    access_log  /var/www/html/test.com/logs/access.log;
    error_log  /var/www/html/test.com/logs/error.log warn;

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}



[root@test ~]# chown -R nginx:nginx /var/www/html/test.com/
[root@test ~]# systemctl restart nginx

 

 

4. SSL Certificate 생성

[root@test ~]# mkdir /etc/ssl/private
[root@test ~]# chmod 700 /etc/ssl/private
[root@test ~]# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/nginx-selfsign.key -out /etc/ssl/certs/nginx-selfsign.crt
Generating a 2048 bit RSA private key
.....................................................................+++
.........................................................................................................................+++
writing new private key to '/etc/ssl/private/nginx-selfsig.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:KR
State or Province Name (full name) []:Seoul
Locality Name (eg, city) [Default City]:GangNam Gu
Organization Name (eg, company) [Default Company Ltd]:test.com
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:test.com
Email Address []:admin@test.com
[root@test ~]#
[root@test ~]# openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
.................................................................
~중략
[root@test ~]#

 

 

5. Nginx 설정 변경

[root@test ~]# vi /etc/nginx/sites-enabled/test_com.conf

server {
    listen       80;
    server_name  www.test.com test.com;
    root   /var/www/html/test.com/public_html;
    index  index.php index.html index.htm;
    location / {
        return 301 https://test.com$request_uri;
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
    }
}

server {
    listen       443 http2 ssl;
    server_name  www.test.com test.com;
    root   /var/www/html/test.com/public_html;
    index  index.php index.html index.htm;

    access_log /var/www/html/test.com/logs/access.log;
    error_log  /var/www/html/test.com/logs/error.log warn;


    ssl_certificate /etc/ssl/certs/nginx-selfsign.crt;
    ssl_certificate_key /etc/ssl/private/nginx-selfsign.key;
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    ssl_session_timeout 10m;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;
    ssl_protocols TLSv1.2;
    ssl_ecdh_curve secp384r1;
    #ssl_ciphers  ECDH+AESGCM:!AES128:!RSA+AES:!aNULL:!MD5:!DSS:!DHE:!kEDH:HIGH:!eNULL:!EXPORT:!DES:!RC4:!PSK:!AECDH:!LOW:!SRP:!ADH:!RSA:TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-256-GCM-SHA384:!COMPLEMENTOFDEFAULT;
    ssl_ciphers AES256+EECDH:AES256+EDH:!aNULL;
    ssl_prefer_server_ciphers on;
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";
#    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;

    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;


    fastcgi_buffering               on;
    fastcgi_buffer_size             16k;
    fastcgi_buffers                 16 16k;

    # time-out settings
    fastcgi_connect_timeout         600s;
    fastcgi_send_timeout            600s;
    fastcgi_read_timeout            600s;

    # php performance settings
    sendfile                        on;
    tcp_nopush                      off;
    keepalive_requests              0;


    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }
    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_read_timeout 600;
        include fastcgi_params;
    }
}
[root@test ~]#
[root@test ~]# systemctl restart nginx
[root@test ~]# vi /var/www/html/test.com/public_html/index.php
<?php phpinfo(); ?>

 

 

6. web-site 접속테스트

 

mysqldump-max_allowed_packet error

max_allowed_packet error 발생시 –max_allowed_packet=1024M 옵션을 사용 하면 됩니다.

[root@localhost ~]# mysqldump -uroot -p --all-databases > db-data.sql
Enter password:
mysqldump: Error 2020: Got packet bigger than 'max_allowed_packet' bytes when dumping table `wp_aiowps_global_meta` at row: 1
[root@localhost ~]# mysqldump -uroot -p --all-databases --max_allowed_packet=1024M > db-data.sql
Enter password:
[root@localhost ~]#

 

 

VM 을 이용한 테스트를 하다 보면 최소 설치후 패키지 설치 하는 작업등 동일한 설치 작업을 많이 합니다.

Vagrantfile 을 이용하면 손쉽게 vm 을 생성 할수 있습니다.

본 문서에서는 Test VM 생성까지만 소개 합니다. 🙂

1. vagrant 명령어

vagrant list-commands 로 vagrant 명령어를 확인 할수 있습니다.

$ vagrant.exe list-commands
Below is a listing of all available Vagrant commands and a brief
description of what they do.

box             manages boxes: installation, removal, etc.
cap             checks and executes capability
cloud           manages everything related to Vagrant Cloud
destroy         stops and deletes all traces of the vagrant machine
docker-exec     attach to an already-running docker container
docker-logs     outputs the logs from the Docker container
docker-run      run a one-off command in the context of a container
global-status   outputs status Vagrant environments for this user
halt            stops the vagrant machine
help            shows the help for a subcommand
init            initializes a new Vagrant environment by creating a Vagrantfile
list-commands   outputs all available Vagrant subcommands, even non-primary ones
login
package         packages a running vagrant environment into a box
plugin          manages plugins: install, uninstall, update, etc.
port            displays information about guest port mappings
powershell      connects to machine via powershell remoting
provider        show provider for this environment
provision       provisions the vagrant machine
push            deploys code in this environment to a configured destination
rdp             connects to machine via RDP
reload          restarts vagrant machine, loads new Vagrantfile configuration
resume          resume a suspended vagrant machine
rsync           syncs rsync synced folders to remote machine
rsync-auto      syncs rsync synced folders automatically when files change
snapshot        manages snapshots: saving, restoring, etc.
ssh             connects to machine via SSH
ssh-config      outputs OpenSSH valid configuration to connect to the machine
status          outputs status of the vagrant machine
suspend         suspends the machine
up              starts and provisions the vagrant environment
upload          upload to machine via communicator
validate        validates the Vagrantfile
vbguest         plugin: vagrant-vbguest: install VirtualBox Guest Additions to the machine
version         prints current and latest Vagrant version
winrm           executes commands on a machine via WinRM
winrm-config    outputs WinRM configuration to connect to the machine

 

 

1.1 vagrant 기본 명령어

일반적으로 많이 사용 하는 명령어를 소개 합니다.

Vagrant 명령어 설명
vagrant init 디렉토리에 Vagrantfile 을 생성 합니다.
vagrant up Vagrantfile 에서 VM 을 생성 합니다.
vagrant ssh vm 연결시 사용 합니다.
vagrant halt vm 을 정지 합니다.
vagrant destroy 생성된 vm 을 삭제 합니다.
vagrant provision vm 에 프로비저닝 합니다.
box add $배포판 Vagrant box 를 다운로드 합니다.
box list 다운로드된 Vagrant box 를 확인 합니다.

 

2. Vagrantfile 작성 참고사항

외부 접속이 필요 없다면 NAT Network 로만 VM 을 구성 하여도 됩니다.

Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"                                           <--- vagrant box 지정 centos / ubuntu 등


network 설정
  config.vm.network "public_network"                                   <--- Bridge Network 사용시 
  config.vm.network "private_network", ip: "192.168.33.10"             <--- NAT Network 사용시


Virtualbox 설정  
   config.vm.provider "virtualbox" do |vb|
   vb.name="CentOS7"                                                    <--- Virtualbox VM name 



프로비저닝 할때는 아래와 같이 vagrant provision 옵션을 사용 합니다. 
sanjuk@testMachine MINGW64 ~/HashiCorp/centos
$ vagrant.exe provision


Provision 설정                                                                   
  config.vm.provision "shell", inline: <<-SHELL
  #   apt-get update
  #   apt-get install -y apache2
  # SHELL
  yum -y update                                                          <---- 설치할 패키지를 지정 
  yum -y install wget
  yum -y install net-tools
  yum -y install bind-utils
  yum -y install epel-release
  wget http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
  rpm -Uvh remi-release-7.rpm
  SHELL
end

 

 

Virtualbox + Vagrant 설치

평소 테스트 환경을 구축 할때 KVM 을 이용하여 virt-clone qemu-img 를 사용하여 Test 머신을 생성 하였습니다.

Vagrant 를 이용하면 vagrantfile 을 이용하여 쉽고 빠르게 테스트 환경을 구성 할수 있습니다.

vmware 의 경우 별도의 라이센스를 구입하여야 사용 가능합니다.

   1. Virtualbox 설치

site: https://www.virtualbox.org/wiki/Downloads   에 접속하여 VirtualBox 를 다운받아 설치 합니다. 

Windows 에서 ssh 를 통한 접속을 위해 VirtualBox Extension Pack 도 설치 합니다.

VirtualBox Extension Pack 의 경우 VirutlaBox 설치후 더블 클릭으로 설치 할수 있습니다.

 

 

   2. Vagrant 설치

site : https://www.vagrantup.com/downloads.html   에서 다운받아 설치를 진행 합니다.

Default 설치를 진행 하였을 경우 c:\HashiCorp 에 설치 됩니다.

터미널은 git bash 를 사용 하였습니다. https://gitforwindows.org/

vagraint init 로 Vagrantfile 을 생성 합니다.

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp
$ mkdir centos

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp
$ cd centos/

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp/centos
$ vagrant init
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp/centos
$

 

   3. Vagrantfile 수정

처음으로 vagrant를 생성 하기 위하여 Vagrantfile 을 수정 합니다.

Vagrant.configure("2") do |config|
  # The most common configuration options are documented and commented below.
  # For a complete reference, please see the online documentation at
  # https://docs.vagrantup.com.

  # Every Vagrant development environment requires a box. You can search for
  # boxes at https://vagrantcloud.com/search.
  config.vm.box = "centos/7"   <-- vm 이미지를 centos7 로 지정 하였습니다.

 

4.1 Vagrant add box

Vagrant box add $boxname 를 하여 사용할 box 를 다운로드 할수 있습니다.

https://app.vagrantup.com/boxes/search site 에서 box 검색할수 있습니다.

# 테스트를 위하여 centos7 박스를 다운로드 합니다. 
$ vagrant box add centos/7
==> box: Loading metadata for box 'centos/7'
    box: URL: https://vagrantcloud.com/centos/7
This box can work with multiple providers! The providers that it
can work with are listed below. Please review the list and choose
the provider you will be working with.

1) hyperv
2) libvirt
3) virtualbox
4) vmware_desktop

Enter your choice: 3
==> box: Adding box 'centos/7' (v1905.1) for provider: virtualbox
    box: Downloading: https://vagrantcloud.com/centos/boxes/7/versions/1905.1/providers/virtualbox.box
==> box: Box download is resuming from prior download progress
    box: Download redirected to host: cloud.centos.org
    box:
==> box: Successfully added box 'centos/7' (v1905.1) for 'virtualbox'!

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp/centos
$ vagrant box list
centos/7 (virtualbox, 1905.1)

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp/centos

 

4.2 Vagrantfile 을 이용한 box 생성

테스트를 위하여 centos7 박스를 생성합니다.

간편하게  vagrant up 명령어를 통하여 VM 을 생성 할수 있습니다.

$ vagrant up

 

4.3 vagrant-vbguest 설치

$ vagrant plugin install vagrant-vbguest
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...
Installed the plugin 'vagrant-vbguest (0.19.0)'!

 

vagrant 생성전

 

vagrant 생성후

 

5. vagrant 접속

vargrant ssh 명령어로 접속 할수 있습니다.

sanjuk@DESKTOP-1HGOOGJ MINGW64 ~/HashiCorp/centos
$ vagrant ssh
[vagrant@localhost ~]$

 

가상머신에서는 vagrant / vagrant 로 접속 하시면 됩니다.

 

6. vagrant 삭제

vagrant destroy 로 vm 을 삭제 할수 있습니다.

$ vagrant destroy
    default: Are you sure you want to destroy the 'default' VM? [y/N] y
==> default: Forcing shutdown of VM...
==> default: Destroying VM and associated drives...

 

테스트 환경 OS: CentOS 7

dr01 active 192.168.122.50 /dev/vdb
dr02 standby 192.168.122.60 /dev/vdb

 

1. hostname 설정 (dr01 , dr02)

[root@dr01 ~]# vi /etc/hosts
192.168.122.50          dr01
192.168.122.60          dr02

 

2. disk 준비 (dr01 , dr02)

[root@drbd01 ~]# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0     11:0    1  4.3G  0 rom
vda    253:0    0   20G  0 disk
├─vda1 253:1    0    1G  0 part /boot
├─vda2 253:2    0    1G  0 part [SWAP]
└─vda3 253:3    0   18G  0 part /
vdb    253:16   0   10G  0 disk
[root@drbd01 ~]#

[root@drbd01 ~]# fdisk /dev/vdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x21877a4c.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-20971519, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519):
Using default value 20971519
Partition 1 of type Linux and of size 10 GiB is set

Command (m for help): wq
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@drbd01 ~]#

 

3. drbd 패키지 설치 (dr01 , dr02)

[root@dr01 ~]# rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
[root@dr01 ~]# rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
Retrieving http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:elrepo-release-7.0-3.el7.elrepo  ################################# [100%]
[root@dr01 ~]#

[root@dr01 ~]# yum install -y kmod-drbd84 drbd84-utils

 

4. drbd 설정 (dr01 , dr02)

[root@dr01 ~]# mv /etc/drbd.d/global_common.conf /etc/drbd.d/global_common.conf.org
[root@dr01 ~]# vi /etc/drbd.d/global_common.conf
global {
 usage-count  yes;
}
common {
 net {
  protocol C;
 }
}


[root@dr01 ~]# vi /etc/drbd.d/dr0.res
resource dr0 {
        on dr01 {
                device /dev/drbd0;
                        disk /dev/vdb1;
                        meta-disk internal;
                        address 192.168.122.50:7789;
        }
        on dr02  {
                device /dev/drbd0;
                        disk /dev/vdb1;
                        meta-disk internal;
                        address 192.168.122.60:7789;
        }
}

 

5. dr0 메터데이터 블록 생성

[root@dr01 ~]# drbdadm create-md dr0
initializing activity log
initializing bitmap (320 KB) to all zero
Writing meta data...
New drbd meta data block successfully created.
[root@dr01 ~]#


[root@dr02 ~]# drbdadm create-md dr0
initializing activity log
initializing bitmap (320 KB) to all zero
Writing meta data...
New drbd meta data block successfully created.
[root@dr02 ~]#

 

6. drbd 데몬 구동 (dr01 , dr02)

[root@dr01 ~]# systemctl start drbd ; systemctl enable drbd
[root@dr02 ~]# systemctl start drbd ; systemctl enable drbd

 

7. drbd primary 설정 (dr01 에서만 실행)

[root@dr01 ~]# drbdadm primary --force dr0

[root@dr01 ~]# drbdadm status dr0
dr0 role:Primary
  disk:UpToDate
  peer role:Secondary
    replication:SyncSource peer-disk:Inconsistent done:0.29

[root@dr01 ~]#

[root@dr01 ~]# drbdadm -- --overwrite-data-of-peer primary dr0

 

8. 동기화 확인 (dr01 , dr02)

[root@dr01 ~]# cat /proc/drbd
version: 8.4.11-1 (api:1/proto:86-101)
GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by mockbuild@, 2018-11-03 01:26:55
 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
    ns:455960 nr:0 dw:0 dr:458080 al:8 bm:0 lo:0 pe:7 ua:0 ap:0 ep:1 wo:f oos:10031116
        [>....................] sync'ed:  4.4% (9796/10236)M
        finish: 0:08:22 speed: 19,972 (13,732) K/sec
[root@dr01 ~]# 


[root@dr02 ~]# cat /proc/drbd
version: 8.4.11-1 (api:1/proto:86-101)
GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by mockbuild@, 2018-11-03 01:26:55
 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----
    ns:0 nr:1981844 dw:1981844 dr:0 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:8502536
        [==>.................] sync'ed: 19.0% (8300/10236)M
        finish: 0:03:48 speed: 37,124 (25,408) want: 41,040 K/sec
[root@dr02 ~]#

 

9. drbd 상태 확인

[root@dr01 ~]# drbdadm status dr0
dr0 role:Primary
  disk:UpToDate
  peer role:Secondary
    replication:Established peer-disk:UpToDate

[root@dr01 ~]#

[root@dr02 ~]# drbdadm status dr0
dr0 role:Secondary
  disk:UpToDate
  peer role:Primary
    replication:Established peer-disk:UpToDate

[root@dr02 ~]#

 

10. 디렉토리 추가 및 파일시스템 포멧

[root@dr01 ~]# mkdir /data
[root@dr02 ~]# mkdir /data

포멧은 dr01 에서만 진행 합니다. 
[root@dr01 ~]# mkfs.xfs /dev/drbd0


drbd0 Device 를 /data 에 마운트 합니다. 
[root@dr01 ~]# mount /dev/drbd0 /data


Test 를 위하여 0 ~ 5 까지 빈파일을 생성 합니다. 
[root@dr01 ~]# cd /data
[root@dr01 data]# touch 0 1 2 3 4 5

 

11.장애 테스트

장애 테스트를 위하여 dr01 시스템을 Down 시킵니다.

dr02 에서 /dev/drbd0 를 /data 에 mount 합니다.

[root@dr01 data]# init 0


[root@dr02 ~]# drbdadm primary dr0
[root@dr02 ~]# mount /dev/drbd0 /data
[root@dr02 ~]# ls -al /data/
total 0
drwxr-xr-x   2 root root  60 Sep  9 14:45 .
dr-xr-xr-x. 18 root root 236 Sep  9 14:42 ..
-rw-r--r--   1 root root   0 Sep  9 14:45 0
-rw-r--r--   1 root root   0 Sep  9 14:45 1
-rw-r--r--   1 root root   0 Sep  9 14:45 2
-rw-r--r--   1 root root   0 Sep  9 14:45 3
-rw-r--r--   1 root root   0 Sep  9 14:45 4
-rw-r--r--   1 root root   0 Sep  9 14:45 5
[root@dr02 ~]#

 

12. dr0 상태를 확인 합니다.

[root@dr02 ~]# drbdadm status dr0
dr0 role:Primary
  disk:UpToDate
  peer connection:Connecting

[root@dr02 ~]#

 

13. 장애 원복

dr02 시스템 에서 primary 를 secondary 로 변경 하고, dr01 시스템을 primary 로 지정 합니다.

[root@dr02 ~]# umount /data
[root@dr02 ~]# drbdadm secondary dr0


[root@dr01 ~]# drbdadm primary dr0
[root@dr01 ~]# mount /dev/drbd0 /data
[root@dr01 ~]# ls -al /data
total 0
drwxr-xr-x   2 root root  60 Sep  9 14:45 .
dr-xr-xr-x. 18 root root 236 Sep  9 14:42 ..
-rw-r--r--   1 root root   0 Sep  9 14:45 0
-rw-r--r--   1 root root   0 Sep  9 14:45 1
-rw-r--r--   1 root root   0 Sep  9 14:45 2
-rw-r--r--   1 root root   0 Sep  9 14:45 3
-rw-r--r--   1 root root   0 Sep  9 14:45 4
-rw-r--r--   1 root root   0 Sep  9 14:45 5
[root@dr01 ~]#

 

14. 상태 확인 

[root@dr01 ~]# drbdadm status dr0
dr0 role:Primary
  disk:UpToDate
  peer role:Secondary
    replication:Established peer-disk:UpToDate

[root@dr01 ~]#




[root@dr02 /]# drbdadm status dr0
dr0 role:Secondary
  disk:UpToDate
  peer role:Primary
    replication:Established peer-disk:UpToDate

[root@dr02 /]#

 

Bash Auto Completion

bash 를 사용하다 보면 tab 을 이용한 자동 완성기능을  많이 사용 합니다.

최소설치로 설치 하였을경우 별도의 패키지를 설치 하여야 합니다.

 

  • 패키지 설치
[root@centos7 ~]# yum install bash-completion bash-completion-extras
[root@centos7 ~]# exit 
재접속후 테스트를 합니다. 

[root@centos7 ~]# ping k8s-
k8s-master   k8s-node01   k8s-node02   k8s-storage

 

 

 

 

  • 의존성 패키지 설치
[root@centos76 ~]# yum install git curl zip unzip

 

  • composer Down
[root@centos76 ~]# curl -sS https://getcomposer.org/installer | php
All settings correct for using Composer
Downloading...

Composer (version 1.9.0) successfully installed to: /root/composer.phar
Use it: php composer.phar
[root@centos76 ~]#

 

  • /usr/bin/composer 로 위치 변경 및 laravel 설치
[root@centos76 ~]# mv composer.phar /usr/bin/composer
[root@centos76 ~]# composer create-project laravel/laravel /var/www/html/laravel

 

  • httpd.conf 수정 및 laravel 권한 수정
[root@centos76 ~]# vi /etc/httpd/conf/httpd.conf
DocumentRoot "/var/www/html/laravel/public"

[root@centos76 ~]# systemctl restart httpd
[root@centos76 ~]# chown -R apache:apache /var/www/html/laravel
[root@centos76 ~]# chmod -R 755 /var/www/html/laravel/storage

 

  • web site 확인

본문서는 작성중인 문서 입니다.  간단하게 nginx-proxy 와 공유디렉토리를 이용하여 wordpress 를 테스트 할수 있습니다.

Test 환경 kvm 에서 Centos7 Version vm 을 3대 준비 합니다.

Nginx-proxy 1대 / LEMP Stack 2대를 구성 합니다.

도메인은 임시로 test.com 도메인을 준비 하였습니다.

구성은 nginx-proxy + LEMP Stack + glusterfs 이며 app 는 WP 가 올라 갑니다.

  • Nginx-proxy 구성도

nginx-proxy

epel-release 패키지 설치 및 nginx 설치

[root@nginx-proxy ~]# yum install epel-release -y

[root@nginx-proxy ~]# vi /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1
[root@nginx-proxy ~]# yum install -y nginx

 

VM nginx-proxy

nginx 설정

[root@nginx-proxy ~]# cd /etc/nginx/conf.d/
[root@nginx-proxy conf.d]# cp default.conf default.conf.org


[root@nginx-proxy conf.d]# cat default.conf
server {
    listen       80;
    server_name  test.com;
    location / {
        rewrite ^/(/.*)$ $1 break; 
        proxy_pass http://test.com;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_redirect off;
    }
}
upstream test.com {
    server 10.10.10.11:80;
    server 10.10.10.22:80;
}

[root@nginx-proxy conf.d]# systemctl enable nginx ; systemctl start nginx

 

VM nginx-www1 / nginx-www2 에서 작업

NGINX , PHP 7.1 을 설치 합니다.

[root@nginx-www1 ~]# vi /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1

[root@nginx-www1 ~]# yum install -y nginx
[root@nginx-www1 ~]# yum install -y epel-release yum-utils
[root@nginx-www1 ~]# rpm -Uvh http://ftp.riken.jp/Linux/remi/enterprise/remi-release-7.rpm
[root@nginx-www1 ~]# yum clean all && yum list
[root@nginx-www1 ~]# yum-config-manager --enable remi-php71
[root@nginx-www1 ~]# yum -y install php php-mysql php-fpm php-opcache php-gd php-ldap \
php-odbc php-pear php-xml php-xmlrpc php-mbstring php-soap curl curl-devel

 

nginx 설정 및 php-fpm 설정

테스트 도메인은 내부 dns 를 구성하여 test.com 으로 생성 하였습니다.

test.com 은 도메인 아이피를 nginx-proxy 로 설정 합니다.

[root@nginx-www1 ~]# mkdir /etc/nginx/sites-enabled
[root@nginx-www1 ~]# vi /etc/nginx/nginx.conf
user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*.conf;
}


[root@nginx-www1 ~]# vi /etc/php-fpm.d/www.conf

user = nginx
group = nginx

listen.owner = nginx
listen.group = nginx
listen.mode = 0660
[root@nginx-www1 ~]# systemctl enable nginx ; systemctl start nginx
[root@nginx-www1 ~]# systemctl enable php-fpm ; systemctl start php-fpm

 

glusterfs 설치

VM nginx-www1 / nginx-www2 에서 작업

/etc/hosts 파일 수정

[root@nginx-www1 ~]# vi /etc/hosts
10.10.10.11     www1
10.10.10.22     www2
10.10.10.33     db01

 

glusterfs 설치

[root@nginx-www1 ~]# yum install centos-release-gluster -y
[root@nginx-www1 ~]# yum install glusterfs-server -y
[root@nginx-www1 ~]# systemctl enable glusterd ; systemctl start glusterd

 

gluster 공유 디렉토리 생성

[root@nginx-www1 ~]# gluster peer probe www2
peer probe: success.
[root@nginx-www2 ~]# gluster peer probe www1



[root@nginx-www1 ~]# mkdir /gluster-storage
[root@nginx-www2 ~]# mkdir /gluster-storage


[root@nginx-www1 ~]# gluster volume create volume01 replica 2 transport tcp www1:/gluster-storage www2:/gluster-storage force
volume create: volume01: success: please start the volume to access data

[root@nginx-www1 ~]# gluster volume start volume01
volume start: volume01: success
[root@nginx-www1 ~]# gluster volume info

Volume Name: volume01
Type: Replicate
Volume ID: b24c3e2b-f458-4733-9bc0-38d9bd441bb6
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: www1:/gluster-storage
Brick2: www2:/gluster-storage
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
[root@nginx-www1 ~]#

[root@nginx-www1 ~]# mkdir -p /var/www/html/test.com/{public_html,logs}

[root@nginx-www1 ~]# vi /etc/fstab

~중략
www1:/volume01  /var/www/html/test.com/public_html glusterfs defaults,_netdev,x-systemd.automount 0 0

[root@nginx-www1 ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda3        18G  1.6G   17G   9% /
devtmpfs        1.9G     0  1.9G   0% /dev
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           1.9G  8.6M  1.9G   1% /run
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/vda1      1014M  215M  800M  22% /boot
tmpfs           379M     0  379M   0% /run/user/0
www1:/volume01   18G  1.8G   17G  10% /var/www/html/test.com/public_html
[root@nginx-www1 ~]#

 

test.com nginx 설정

[root@nginx-www1 ~]# vi /etc/nginx/sites-enabled/test_com.conf
server {
    listen       80;
    server_name  www.test.com test.com;
    root   /var/www/html/test.com/public_html;
    index  index.php index.html index.htm;
    location / {
        try_files $uri $uri/ /index.php?$query_string;
        autoindex on;
    }

    access_log  /var/www/html/test.com/logs/access.log;
    error_log  /var/www/html/test.com/logs/error.log warn;

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

 

nginx 데몬 재시작 및 phpinfo 확인

[root@nginx-www1 ~]# systemctl restart nginx
[root@nginx-www1 ~]# vi /var/www/html/test.com/public_html/info.php
<?php phpinfo(); ?>

 

www2 시스템 에서 test.com public_html 디렉토리 확인시 정상적으로 info.php 를 확인 할수 있습니다.

[root@nginx-www2 ~]# ls -al /var/www/html/test.com/public_html/
total 1
drwxr-xr-x 3 nginx nginx 40 Jul 26 14:37 .
drwxr-xr-x 4 nginx nginx 37 Jul 26 14:33 ..
-rw-r--r-- 1 root  root  20 Jul 26 14:37 info.php
[root@nginx-www2 ~]#

 

phpinfo 확인

 

Mariadb 10.1 설치

별도의 vm 에 db 를 설치 합니다.

[root@nginx-mariadb01 ~]# vi /etc/yum.repos.d/mariadb.repo
[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/10.1/centos7-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1


[root@nginx-mariadb01 ~]# yum install -y mariadb mariadb-server
[root@nginx-mariadb01 ~]# systemctl start mariadb ; systemctl enable mariadb

 

mysql_secure_installation 을 실행 합니다.

[root@nginx-mariadb01 ~]# /usr/bin/mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
      SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!

In order to log into MariaDB to secure it, we'll need the current
password for the root user.  If you've just installed MariaDB, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none):
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.

Set root password? [Y/n] y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..
 ... Success!


By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] y
 ... Success!

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] y
 ... Success!

By default, MariaDB comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] y
 - Dropping test database...
 ... Success!
 - Removing privileges on test database...
 ... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] y
 ... Success!

Cleaning up...

All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.

Thanks for using MariaDB!
[root@nginx-mariadb01 ~]#

 

character set 설정

[root@nginx-mariadb01 ~]# vi /etc/my.cnf.d/server.cnf
[mysqld]
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

[root@nginx-mariadb01 ~]# vi /etc/my.cnf.d/client.cnf
[client]
default-character-set = utf8mb4

 

character set 확인

[root@nginx-mariadb01 ~]#  systemctl restart mariadb
[root@nginx-mariadb01 ~]# mysql -uroot -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 2
Server version: 10.1.40-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> status;
--------------
mysql  Ver 15.1 Distrib 10.1.40-MariaDB, for Linux (x86_64) using readline 5.1

Connection id:          2
Current database:
Current user:           root@localhost
SSL:                    Not in use
Current pager:          stdout
Using outfile:          ''
Using delimiter:        ;
Server:                 MariaDB
Server version:         10.1.40-MariaDB MariaDB Server
Protocol version:       10
Connection:             Localhost via UNIX socket
Server characterset:    utf8mb4
Db     characterset:    utf8mb4
Client characterset:    utf8mb4
Conn.  characterset:    utf8mb4
UNIX socket:            /var/lib/mysql/mysql.sock
Uptime:                 21 sec

Threads: 1  Questions: 4  Slow queries: 0  Opens: 17  Flush tables: 1  Open tables: 11  Queries per second avg: 0.190
--------------

MariaDB [(none)]> quit;
Bye
[root@nginx-mariadb01 ~]#

 

WordPress  에서 사용할 DB 를 생성 합니다.

[root@nginx-mariadb01 ~]# mysql -uroot -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 10.1.40-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> create database wp;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> GRANT ALL ON wp.* TO 'wp'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> quit;
Bye
[root@nginx-mariadb01 ~]#

 

www1 / www2 에 hosts 파일을 수정 합니다.

wordpress 설치 파일은 www1 에서만 작업 합니다.

[root@nginx-www1 ~]# vi /etc/hosts

10.10.10.11     www1
10.10.10.22     www2
10.10.10.33     db01

 

WordPress 설치

[root@nginx-www1 ~]# cd /var/www/html/test.com/public_html/
[root@nginx-www1 public_html]# wget https://wordpress.org/latest.tar.gz
[root@nginx-www1 public_html]# tar xvf latest.tar.gz
[root@nginx-www1 public_html]# cd wordpress/
[root@nginx-www1 wordpress]# mv * ../
[root@nginx-www1 public_html]# rm -rf wordpress/

 

test.com 사이트에 접속하여 워드프레스를 설치 합니다.

 

database 정보 입력

 

Run installation

 

test.com site 정보 입력

 

워드프레스 설치가 완료 되었습니다.

 

test.com 으로 접속시 nginx-proxy 를 통하여 www1 / www2 로 접속을 합니다.

 

로그 확인시 www1 / www2 에 한번씩 접속 로그가 생성 됩니다.

[CentOS7] Redis 설치 및 php연동

LEMP Stack 의 경우 링크 사이트를 참고해 주세요. http://docs.crois.net/linux/linux/#lemp-stack   

LEMP Stack 이후 부터 설치 하시면 됩니다.

Source 로 설치 하는 방법도 있지만 간단하게 yum 으로 설치 하는 방법을 기술 합니다.

Redis 의 자세한 내용은 차후 정리 하도록 하겠습니다.

 

  • Redis 설치
[root@centos-nginx ~]# yum install -y redis php71-php-pecl-redis php71-php-phpiredis php-redis

 

  • 설치확인
[root@centos-nginx ~]# php -i |grep redis
/etc/php.d/50-redis.ini
redis
redis.arrays.algorithm => no value => no value
redis.arrays.auth => no value => no value
redis.arrays.autorehash => 0 => 0
redis.arrays.connecttimeout => 0 => 0
redis.arrays.consistent => 0 => 0
redis.arrays.distributor => no value => no value
redis.arrays.functions => no value => no value
redis.arrays.hosts => no value => no value
redis.arrays.index => 0 => 0
redis.arrays.lazyconnect => 0 => 0
redis.arrays.names => no value => no value
redis.arrays.pconnect => 0 => 0
redis.arrays.previous => no value => no value
redis.arrays.readtimeout => 0 => 0
redis.arrays.retryinterval => 0 => 0
redis.clusters.auth => no value => no value
redis.clusters.cache_slots => 0 => 0
redis.clusters.persistent => 0 => 0
redis.clusters.read_timeout => 0 => 0
redis.clusters.seeds => no value => no value
redis.clusters.timeout => 0 => 0
redis.pconnect.connection_limit => 0 => 0
redis.pconnect.pooling_enabled => 1 => 1
redis.session.lock_expire => 0 => 0
redis.session.lock_retries => 10 => 10
redis.session.lock_wait_time => 2000 => 2000
redis.session.locking_enabled => 0 => 0
Registered save handlers => files user redis rediscluster
This program is free software; you can redistribute it and/or modify
[root@centos-nginx ~]#

 

  • php-fpm 데몬 재시작
[root@centos-nginx ~]# systemctl restart php-fpm

 

  • phpinfo 페이지 확인