CentOS7 minikube 설치

참고 페이지 : https://kubernetes.io/ko/docs/tasks/tools/install-minikube/

https://computingforgeeks.com/how-to-run-minikube-on-kvm/

kvm 환경에서 minikube 설치 방법을 정리 하였습니다. Virtualbox 및 vmware 에서는 kubernetes 3 node 구성이 좀더 편하게 설치 됩니다. 🙂

 

 

1. 가상화 지원 확인

[root@kvm-server01 ~]# grep -E --color 'vmx|svm' /proc/cpuinfo
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall
nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cp
l vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt aes lahf_lm ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid
dtherm ida arat spec_ctrl intel_stibp flush_l1d
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall
nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cp
l vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt aes lahf_lm ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid
dtherm ida arat spec_ctrl intel_stibp flush_l1d
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall
nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cp
l vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt aes lahf_lm ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid
dtherm ida arat spec_ctrl intel_stibp flush_l1d
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall
nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cp
l vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt aes lahf_lm ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid
dtherm ida arat spec_ctrl intel_stibp flush_l1d
~ 중략

 

 

2. Download minikube

[root@kvm-server01 ~]# wget https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
[root@kvm-server01 ~]# chmod +x minikube-linux-amd64
[root@kvm-server01 ~]# mv minikube-linux-amd64 /usr/local/bin/minikube
[root@kvm-server01 ~]# minikube version
minikube version: v1.3.1
commit: ca60a424ce69a4d79f502650199ca2b52f29e631
[root@kvm-server01 ~]#

 

 

3. Install kubectl

[root@kvm-server01 ~]# curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 40.9M 100 40.9M 0 0 29.0M 0 0:00:01 0:00:01 --:--:-- 29.0M
[root@kvm-server01 ~]# chmod +x kubectl
[root@kvm-server01 ~]# mv kubectl /usr/local/bin/
[root@kvm-server01 ~]# kubectl version -o json
{
"clientVersion": {
"major": "1",
"minor": "15",
"gitVersion": "v1.15.3",
"gitCommit": "2d3c76f9091b6bec110a5e63777c332469e0cba2",
"gitTreeState": "clean",
"buildDate": "2019-08-19T11:13:54Z",
"goVersion": "go1.12.9",
"compiler": "gc",
"platform": "linux/amd64"
}
}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[root@kvm-server01 ~]#

 

 

4. Install Docker Machine KVM Driver

[root@kvm-server01 ~]# curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13.8M 100 13.8M 0 0 13.4M 0 0:00:01 0:00:01 --:--:-- 13.5M
[root@kvm-server01 ~]#
[root@kvm-server01 ~]# chmod +x docker-machine-driver-kvm2
[root@kvm-server01 ~]# mv docker-machine-driver-kvm2 /usr/local/bin/

Starting minikube

[root@kvm-server01 ~]# chmod +x docker-machine-driver-kvm2
[root@kvm-server01 ~]# mv docker-machine-driver-kvm2 /usr/local/bin/
[root@kvm-server01 ~]# minikube start --vm-driver kvm2
* minikube v1.3.1 on Centos 7.6.1810
! Please don't run minikube as root or with 'sudo' privileges. It isn't necessary with kvm2 driver.
* Downloading VM boot image ...
minikube-v1.3.0.iso.sha256: 65 B / 65 B [--------------------] 100.00% ? p/s 0s
minikube-v1.3.0.iso: 131.07 MiB / 131.07 MiB [-------] 100.00% 48.99 MiB p/s 3s
* Creating kvm2 VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
* Preparing Kubernetes v1.15.2 on Docker 18.09.8 ...
* Downloading kubeadm v1.15.2
* Downloading kubelet v1.15.2
* Pulling images ...
* Launching Kubernetes ...
* Waiting for: apiserver proxy etcd scheduler controller dns
* Done! kubectl is now configured to use "minikube"
[root@kvm-server01 ~]#


[root@kvm-server01 ~]# sudo virsh list |grep mini
320 minikube running
[root@kvm-server01 ~]#

[root@kvm-server01 ~]# kubectl cluster-info
Kubernetes master is running at https://192.168.39.228:8443
KubeDNS is running at https://192.168.39.228:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@kvm-server01 ~]#


[root@kvm-server01 ~]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /root/.minikube/ca.crt
server: https://192.168.39.228:8443
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /root/.minikube/client.crt
client-key: /root/.minikube/client.key
[root@kvm-server01 ~]#


[root@kvm-server01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready master 15m v1.15.2
[root@kvm-server01 ~]# minikube ssh
_ _
_ _ ( ) ( )
___ ___ (_) ___ (_)| |/') _ _ | |_ __
/' _ ` _ `\| |/' _ `\| || , < ( ) ( )| '_`\ /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )( ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$ cat /etc/os-release
NAME=Buildroot
VERSION=2018.05.3
ID=buildroot
VERSION_ID=2018.05.3
PRETTY_NAME="Buildroot 2018.05.3"
$

 

 

VM 을 이용한 테스트를 하다 보면 최소 설치후 패키지 설치 하는 작업등 동일한 설치 작업을 많이 합니다.

Vagrantfile 을 이용하면 손쉽게 vm 을 생성 할수 있습니다.

본 문서에서는 Test VM 생성까지만 소개 합니다. 🙂

1. vagrant 명령어

vagrant list-commands 로 vagrant 명령어를 확인 할수 있습니다.

$ vagrant.exe list-commands
Below is a listing of all available Vagrant commands and a brief
description of what they do.

box             manages boxes: installation, removal, etc.
cap             checks and executes capability
cloud           manages everything related to Vagrant Cloud
destroy         stops and deletes all traces of the vagrant machine
docker-exec     attach to an already-running docker container
docker-logs     outputs the logs from the Docker container
docker-run      run a one-off command in the context of a container
global-status   outputs status Vagrant environments for this user
halt            stops the vagrant machine
help            shows the help for a subcommand
init            initializes a new Vagrant environment by creating a Vagrantfile
list-commands   outputs all available Vagrant subcommands, even non-primary ones
login
package         packages a running vagrant environment into a box
plugin          manages plugins: install, uninstall, update, etc.
port            displays information about guest port mappings
powershell      connects to machine via powershell remoting
provider        show provider for this environment
provision       provisions the vagrant machine
push            deploys code in this environment to a configured destination
rdp             connects to machine via RDP
reload          restarts vagrant machine, loads new Vagrantfile configuration
resume          resume a suspended vagrant machine
rsync           syncs rsync synced folders to remote machine
rsync-auto      syncs rsync synced folders automatically when files change
snapshot        manages snapshots: saving, restoring, etc.
ssh             connects to machine via SSH
ssh-config      outputs OpenSSH valid configuration to connect to the machine
status          outputs status of the vagrant machine
suspend         suspends the machine
up              starts and provisions the vagrant environment
upload          upload to machine via communicator
validate        validates the Vagrantfile
vbguest         plugin: vagrant-vbguest: install VirtualBox Guest Additions to the machine
version         prints current and latest Vagrant version
winrm           executes commands on a machine via WinRM
winrm-config    outputs WinRM configuration to connect to the machine

 

 

1.1 vagrant 기본 명령어

일반적으로 많이 사용 하는 명령어를 소개 합니다.

Vagrant 명령어 설명
vagrant init 디렉토리에 Vagrantfile 을 생성 합니다.
vagrant up Vagrantfile 에서 VM 을 생성 합니다.
vagrant ssh vm 연결시 사용 합니다.
vagrant halt vm 을 정지 합니다.
vagrant destroy 생성된 vm 을 삭제 합니다.
vagrant provision vm 에 프로비저닝 합니다.
box add $배포판 Vagrant box 를 다운로드 합니다.
box list 다운로드된 Vagrant box 를 확인 합니다.

 

2. Vagrantfile 작성 참고사항

외부 접속이 필요 없다면 NAT Network 로만 VM 을 구성 하여도 됩니다.

Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"                                           <--- vagrant box 지정 centos / ubuntu 등


network 설정
  config.vm.network "public_network"                                   <--- Bridge Network 사용시 
  config.vm.network "private_network", ip: "192.168.33.10"             <--- NAT Network 사용시


Virtualbox 설정  
   config.vm.provider "virtualbox" do |vb|
   vb.name="CentOS7"                                                    <--- Virtualbox VM name 



프로비저닝 할때는 아래와 같이 vagrant provision 옵션을 사용 합니다. 
sanjuk@testMachine MINGW64 ~/HashiCorp/centos
$ vagrant.exe provision


Provision 설정                                                                   
  config.vm.provision "shell", inline: <<-SHELL
  #   apt-get update
  #   apt-get install -y apache2
  # SHELL
  yum -y update                                                          <---- 설치할 패키지를 지정 
  yum -y install wget
  yum -y install net-tools
  yum -y install bind-utils
  yum -y install epel-release
  wget http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
  rpm -Uvh remi-release-7.rpm
  SHELL
end

 

 

Virtualbox + Vagrant 설치

평소 테스트 환경을 구축 할때 KVM 을 이용하여 virt-clone qemu-img 를 사용하여 Test 머신을 생성 하였습니다.

Vagrant 를 이용하면 vagrantfile 을 이용하여 쉽고 빠르게 테스트 환경을 구성 할수 있습니다.

vmware 의 경우 별도의 라이센스를 구입하여야 사용 가능합니다.

   1. Virtualbox 설치

site: https://www.virtualbox.org/wiki/Downloads   에 접속하여 VirtualBox 를 다운받아 설치 합니다. 

Windows 에서 ssh 를 통한 접속을 위해 VirtualBox Extension Pack 도 설치 합니다.

VirtualBox Extension Pack 의 경우 VirutlaBox 설치후 더블 클릭으로 설치 할수 있습니다.

 

 

   2. Vagrant 설치

site : https://www.vagrantup.com/downloads.html   에서 다운받아 설치를 진행 합니다.

Default 설치를 진행 하였을 경우 c:\HashiCorp 에 설치 됩니다.

터미널은 git bash 를 사용 하였습니다. https://gitforwindows.org/

vagraint init 로 Vagrantfile 을 생성 합니다.

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp
$ mkdir centos

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp
$ cd centos/

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp/centos
$ vagrant init
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp/centos
$

 

   3. Vagrantfile 수정

처음으로 vagrant를 생성 하기 위하여 Vagrantfile 을 수정 합니다.

Vagrant.configure("2") do |config|
  # The most common configuration options are documented and commented below.
  # For a complete reference, please see the online documentation at
  # https://docs.vagrantup.com.

  # Every Vagrant development environment requires a box. You can search for
  # boxes at https://vagrantcloud.com/search.
  config.vm.box = "centos/7"   <-- vm 이미지를 centos7 로 지정 하였습니다.

 

4.1 Vagrant add box

Vagrant box add $boxname 를 하여 사용할 box 를 다운로드 할수 있습니다.

https://app.vagrantup.com/boxes/search site 에서 box 검색할수 있습니다.

# 테스트를 위하여 centos7 박스를 다운로드 합니다. 
$ vagrant box add centos/7
==> box: Loading metadata for box 'centos/7'
    box: URL: https://vagrantcloud.com/centos/7
This box can work with multiple providers! The providers that it
can work with are listed below. Please review the list and choose
the provider you will be working with.

1) hyperv
2) libvirt
3) virtualbox
4) vmware_desktop

Enter your choice: 3
==> box: Adding box 'centos/7' (v1905.1) for provider: virtualbox
    box: Downloading: https://vagrantcloud.com/centos/boxes/7/versions/1905.1/providers/virtualbox.box
==> box: Box download is resuming from prior download progress
    box: Download redirected to host: cloud.centos.org
    box:
==> box: Successfully added box 'centos/7' (v1905.1) for 'virtualbox'!

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp/centos
$ vagrant box list
centos/7 (virtualbox, 1905.1)

sanjuk@DESKTOP-O1Q8NLC MINGW64 /c/HashiCorp/centos

 

4.2 Vagrantfile 을 이용한 box 생성

테스트를 위하여 centos7 박스를 생성합니다.

간편하게  vagrant up 명령어를 통하여 VM 을 생성 할수 있습니다.

$ vagrant up

 

4.3 vagrant-vbguest 설치

$ vagrant plugin install vagrant-vbguest
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...
Installed the plugin 'vagrant-vbguest (0.19.0)'!

 

vagrant 생성전

 

vagrant 생성후

 

5. vagrant 접속

vargrant ssh 명령어로 접속 할수 있습니다.

sanjuk@DESKTOP-1HGOOGJ MINGW64 ~/HashiCorp/centos
$ vagrant ssh
[vagrant@localhost ~]$

 

가상머신에서는 vagrant / vagrant 로 접속 하시면 됩니다.

 

6. vagrant 삭제

vagrant destroy 로 vm 을 삭제 할수 있습니다.

$ vagrant destroy
    default: Are you sure you want to destroy the 'default' VM? [y/N] y
==> default: Forcing shutdown of VM...
==> default: Destroying VM and associated drives...