2020 on Ricky

WSL 2 .wslconfig configuration explained

Wed, 30 Dec 2020 21:24:12 +0800

Steps to install WSL 2:

Join the Windows Insider Program (required)
Enable required WSL components
Install a Linux distribution
Set the Linux distribution to use WSL 2
WSL 2 troubleshooting: compressed virtual disk files cannot be converted to the WSL 2 architecture
Install and start Docker
Install Docker Desktop v2.2.1.0

# Enable required WSL components
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

# Set the Linux distribution to WSL 2
wsl --set-version ubuntu 2
wsl --set-default-version 2

Edit %UserProfile%\.wslconfig

Day 19 BGP Protocol (1)

Wed, 30 Dec 2020 20:00:33 +0800

Style PowerShell with oh-my-posh

Thu, 10 Dec 2020 13:15:59 +0800

Style PowerShell with oh-my-posh

# This downloads and installs the posh-git and oh-my-posh modules from PowerShell Gallery.
# The former shows Git info in the prompt, and the latter provides the themes.
Install-Module posh-git -Scope CurrentUser
Install-Module oh-my-posh -Scope CurrentUser


# Next, edit the PowerShell profile loaded at startup. In PowerShell, $PROFILE shows the current
# user's profile path. The file may not exist; run the commands below to create it and open it.
if (!(Test-Path -Path $PROFILE )) { New-Item -Type File -Path $PROFILE -Force }
notepad $PROFILE

Add the following commands to the profile file:

An introduction to hacker tools: the tip of the iceberg

Tue, 08 Dec 2020 21:50:47 +0800

An introduction to hacker tools: the tip of the iceberg

Provisioning a Windows Server Vagrant box with IIS, .NET 4.5 and Octopus Deploy

Thu, 03 Dec 2020 12:39:01 +0800

New LibSSH Connection Plugin for Ansible Network Replaces Paramiko, Adds FIPS Mode Enablement

Wed, 25 Nov 2020 21:09:50 +0800

New LibSSH Connection Plugin for Ansible Network Replaces Paramiko, Adds FIPS Mode Enablement

Switching Ansible Playbooks to use LibSSH

# Installing LibSSH
pip install ansible-pylibssh

Using LibSSH in Ansible Playbooks

Method 1. The ssh_type configuration parameter can be set to use libssh in the active ansible.cfg file of your project

[persistent_connection]
ssh_type = libssh

Method 2: Set the ANSIBLE_NETWORK_CLI_SSH_TYPE environment variable

$ export ANSIBLE_NETWORK_CLI_SSH_TYPE=libssh

Method 3: Set the ansible_network_cli_ssh_type parameter to libssh within your playbook at the play level

Mount a Synology NAS folder on CentOS 7

Mon, 09 Nov 2020 12:12:32 +0800

Mount a Synology NAS folder on CentOS 7

Set SSLKEYLOGFILE on MacBook to decrypt HTTPS traffic

Fri, 06 Nov 2020 20:02:54 +0800

Set SSLKEYLOGFILE on MacBook to decrypt HTTPS traffic

Create keylogfile

mkdir ~/sslkeylogfile && touch ~/sslkeylogfile/keylogfile.log # create keylogfile.log
sudo chmod 777 ~/sslkeylogfile/keylogfile.log # change permissions so Chrome can write on startup

Configure environment variable

vim ~/.zshrc # open config file
export SSLKEYLOGFILE=~/sslkeylogfile/keylogfile.log # set environment variable
source ~/.zshrc # reload config

Configure Wireshark

preferences -> Protocols -> TLS

Set TLS debug file to record decryption logs
Set (Pre)-Master-Secret log filename to the absolute path of keylogfile.log

Start Chrome from terminal

Backup FortiOS config with Ansible - with RestAPI

Tue, 03 Nov 2020 17:59:24 +0800

Create access profile

FGTAWS0004BE1ADE # config system accprofile
FGTAWS0004BE1ADE (accprofile) # edit readOnly
new entry 'readOnly' added
FGTAWS0004BE1ADE (readOnly) # set sysgrp read
FGTAWS0004BE1ADE (readOnly) # end

Create API user in Fortigate

FGTAWS0004BE1ADE # config system api-user
FGTAWS0004BE1ADE (api-user) # edit api-admin
new entry 'api-admin' added
FGTAWS0004BE1ADE (api-admin) # set accprofile "readOnly"
FGTAWS0004BE1ADE (api-admin) # set vdom root
FGTAWS0004BE1ADE (api-admin) # config trusthost
FGTAWS0004BE1ADE (trusthost) # edit 1
new entry '1' added
FGTAWS0004BE1ADE (1) # set ipv4-trusthost 'ip_address_of_your_machine' 255.255.255.255
FGTAWS0004BE1ADE (1) # end
FGTAWS0004BE1ADE (api-admin) # end

Generate API token

FGTAWS0004BE1ADE # execute api-user generate-key api-admin
New API key: 'your_api_token'
NOTE: The bearer of this API key will be granted all access privileges assigned to the api-user api-admin.

Test

# fortigate.py
import requests
import urllib3 # disable security warning for SSL certificate
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) # disable security warning for SSL certificate


def config_download(ipaddr, api_token, filename='backup.conf'):
 '''
 input: ipaddr(string) - target ip address of fortigate
 input: api_token(string) - api_token for api user(accprofile should have sysgrp.mnt)
 input: filename(string) - file name of the config to be saved. default backup.conf
 output: True if backup successfule. False if not successful.
 Tested on: Fortigate OnDemand on AWS - FortiOS6.0.4
 '''
 base_url = f'https://{ipaddr}/api/v2/'
 headers = {'Authorization': f'Bearer {api_token}'}
 params = {'scope': 'global'}
 uri = 'monitor/system/config/backup/'

 rep = requests.get(base_url + uri, headers=headers, params=params, verify=False)

 if rep.status_code != 200:
 print(f'Something went wrong. status_code: {rep.status_code}')
 return False

 with open(filename, 'w') as f:
 f.write(rep.text)

 return True

>>> import fortigate
>>>
>>> ip_addr = 'Fortigate_IP_Address'
>>> api_token = 'API_TOKEN'
>>>
>>> if (fortigate.config_download(ip_addr, api_token, 'backup20190215.conf')):
... print('Done!')
... else:
... print('Error!!')
...
Done!
>>>
>>> with open('backup20190215.conf', 'r') as f:
... f.readline()
...
'#config-version=FGTAWS-6.0.4-FW-build0231-190107:opmode=0:vdom=0:user=api-admin\n'
>>>

Configure Ansible inventory and playbook

$ cat hosts
[fortigate]
x.x.x.x access_token=w4q9qtfbGry3Nbc40kHjsk9mxG****
y.y.y.y access_token=tfy8c9b8Nxw6N3Q5Q5bg9z69dn****

$ cat fortigate_backup.yml
 - name: fortigate config backup
 connection: local
 hosts: fortigate

 tasks:
 - name: get current config
 uri:
 url: 'https://{{ ansible_host }}/api/v2/monitor/system/config/backup/?scope=global&access_token={{ access_token }}'
 return_content: yes
 validate_certs: no
 register: current_config

 - name: write config to local file
 local_action: copy content={{ current_config.content }} dest=./{{ inventory_hostname }}_{{ ansible_date_time.date }}.txt

In 2020, use the latest NGINX ngx_http_geoip2 module to block IPs by country or region

Tue, 27 Oct 2020 15:44:48 +0800

Install geoip2 lib

cd /usr/local/src
rm -f libmaxminddb-1.4.2.tar.gz
wget https://github.com/maxmind/libmaxminddb/releases/download/1.4.2/libmaxminddb-1.4.2.tar.gz
tar -xzf libmaxminddb-1.4.2.tar.gz
cd libmaxminddb-1.4.2
yum install gcc gcc-c++ make -y
./configure
make
make check
sudo make install

echo '/usr/local/lib' > /etc/ld.so.conf.d/geoip.conf
sudo ldconfig

Download ngx_http_geoip2_module

cd /usr/local/src
wget https://github.com/leev/ngx_http_geoip2_module/archive/3.3.tar.gz
tar -xzf 3.3.tar.gz
mv ngx_http_geoip2_module-3.3 ngx_http_geoip2_module

# nginx集成
cd /usr/local/src
wget http://nginx.org/download/nginx-1.16.1.tar.gz
tar -zxf nginx-1.16.1.tar.gz
cd nginx-1.16.1
useradd -M -s /sbin/nologin www


yum install gcc gcc-c++ make pcre-devel zlib-devel openssl-devel -y
./configure --user=www --group=www --prefix=/usr/local/nginx \
--with-ld-opt="-Wl,-rpath -Wl,/usr/local/lib" \
--with-http_sub_module \
--with-http_realip_module \
--with-http_gzip_static_module \
--with-http_ssl_module \
--with-http_v2_module \
--add-module=/usr/local/src/ngx_http_geoip2_module

make
make install

Download geoip2 IP database

The latest GeoLite2-City.mmdb for 2020 cannot be downloaded directly. You must register a maxmind account.

Nginx HTTPS with Basic Auth reverse proxy for VMware ESXi 6.5 fixed VMRC /screen

Sat, 17 Oct 2020 12:31:02 +0800

Nginx HTTPS with Basic Auth reverse proxy for VMware ESXi 6.5 fixed VMRC /screen

server {
 listen 80;
 server_name esxi.hackion.com;
 return 301 https://$server_name$request_uri;
}

server {
 listen 443 ssl;
 server_name esxi.hackion.com;

 ssl_certificate /mycert.crt;
 ssl_certificate_key /mykey.key;

 location / {
 auth_basic "Restricted Content";
 auth_basic_user_file /etc/nginx/.htpasswd;

 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header Origin '';
 proxy_set_header Authorization ''; #Don't pass the Nginx Basic Auth to ESXi or it will break VMRC.
 proxy_pass_header X-XSRF-TOKEN;
 proxy_pass https://esxi_server;
 proxy_send_timeout 300;
 proxy_read_timeout 300;
 send_timeout 300;
 client_max_body_size 1000m;

 # enables WS support
 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection "upgrade";
 }
}

server {
 listen 443 ssl http2;

 # ssl_certificate and ssl_certificate_key are required
 ssl_certificate /etc/letsencrypt/live/myletsencryptdomain/fullchain.pem;
 ssl_certificate_key /etc/letsencrypt/live/myletsencryptdomain/privkey.pem;

 include /etc/nginx/snippets/ssl-params.conf;
 # removed DH params as my ssl-params.conf specifies to only use ECDHE key exchange.

 server_name fqdn.extern;

 location / {
 proxy_set_header Host $http_host;
 proxy_set_header X-Real-IP $remote_addr;

 proxy_ssl_verify off; # No need on isolated LAN
 proxy_pass https://vcenter.ip; # esxi IP Address

 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection "upgrade";

 proxy_buffering off;
 client_max_body_size 0;
 proxy_read_timeout 36000s;

 proxy_redirect https://fqdn.local/ https://fqdn.extern/; # read comment below
 # replace vcenter-hostname with your actual vcenter's hostname, and esxi with your nginx's server_name.
 }

 location /websso/SAML2 {
 proxy_set_header Host fqdn.local; # your actual vcenter's hostname
 proxy_set_header X-Real-IP $remote_addr;

 proxy_ssl_verify off; # No need on isolated LAN
 proxy_pass https://vcenter.ip; # esxi IP Address

 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection "upgrade";

 proxy_buffering off;
 client_max_body_size 0;
 proxy_read_timeout 36000s;
 proxy_ssl_session_reuse on;

 proxy_redirect https://fqdn.local/ https://fqdn.extern/; # read comment below
 # replace vcenter-hostname with your actual vcenter's hostname, and esxi with your nginx's server_name.
 }
}

Running GitHub Actions for Certain Commit Messages

Sun, 11 Oct 2020 23:13:22 +0800

Running GitHub Actions for Certain Commit Messages

Now, whenever I push a wip commit or any commit that contains the word wip, it will be marked as skipped inside of GitHub actions.

jobs:
 format:
 runs-on: ubuntu-latest
 if: "! contains(github.event.head_commit.message, 'wip')"

Any commit that contains [build] will now trigger these jobs, everything else will be skipped.

jobs:
 format:
 runs-on: ubuntu-latest
 if: "contains(github.event.head_commit.message, '[build]')"

Using Vagrant to Deploy Multiple VMs on vSphere

Mon, 05 Oct 2020 09:53:03 +0800

Using Vagrant to Deploy Multiple VMs on vSphere

vm1 = { 'name' => "PhotonVM1", 'ip' => "192.168.5.224" }
vm2 = { 'name' => "PhotonVM2", 'ip' => "192.168.5.225" }
vms = [ vm1, vm2]

Vagrant.configure(2) do |config|
 vms.each do |node|
 vm_name = node['name']
 vm_ip = node['ip']
 config.vm.define vm_name do |node_config|
 node_config.vm.network 'private_network', ip: "#{vm_ip}"
 node_config.vm.box = "dummy"
 node_config.vm.box_url = "./example_box/dummy.box"
 node_config.vm.provider :vsphere do |vsphere|
 vsphere.host = 'vc01.testlab.local'
 vsphere.compute_resource_name = 'esxi01.testlab.local'
 vsphere.name = vm_name
 vsphere.customization_spec_name = 'centos66'
 vsphere.template_name = 'PhotonTemplate'
 vsphere.user = 'administrator@vsphere.local'
 vsphere.password = 'password'
 vsphere.insecure = true
 end
 end
 end
end

Vagrantfile and Provider

Fri, 02 Oct 2020 19:41:23 +0800

用以下這個範例你只要使用 Vagrant up 加上 provider 參數，就可開啟不同來源的機器

例如我要開啟 Vsphere 的機器，我只要下 vagrant up --provider=vsphere

要開 AWS 的機器，我只要下 vagrant up --provider=aws

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
 # 我們定義一個 Ubuntu 的機器
 # box 的需求是對於 local 的 VM 才需要的
 # 所以在 provider 是 vmware_fusion 或 Virtualbox 時，才會用到這個設定
 config.vm.box = "hashicorp/precise64"

 # vmware_fusion
 config.vm.provider "vmware_fusion" do |v, override|
 override.vm.box = "precise64_vmware"
 v.gui = false
 end

 # vsphere
 config.vm.provider :vsphere do |vsphere|
 # 對於私有雲及公有雲，要clone 的vm 是儲存在雲端上的
 # 所以 box 使用 dummy box 來達到這個目的
 vsphere.vm.box = "nkhasanov/vsphere-simple"
 vsphere.host = 'YOURIP'
 vsphere.compute_resource_name = 'YOUR DATACENTER'
 vsphere.resource_pool_name = 'YOUR RESOURCE POOL'
 vsphere.insecure = true
 vsphere.template_name = 'VM TEMPLATE'
 vsphere.name = "#{YOUR_NAME}-test-machine"
 vsphere.user = 'administrator'
 vsphere.password = '$p1unK_Lab'
 vsphere.vm_base_path = 'vmware_template'
 vsphere.linked_clone = true
 end

 # virtual box
 config.vm.provider "virtualbox" do |vb|
 vb.gui = false
 end

 # aws
 config.vm.provider :aws do |aws, override|
 # aws configurations
 aws.access_key_id = "#{YOUR_AWS_ACCESS_KEY_ID}"
 aws.secret_access_key = "#{YOUR_AWS_ACCESS_KEY}"
 aws.keypair_name = "#{YOUR_NAME}"
 aws.security_groups = "#{YOUR_NAME}"
 aws.instance_type = "t2.small"
 aws.region = "us-east-1"
 # ubuntu 14.04 x64
 aws.ami = "ami-864d84ee"

 # override info
 override.ssh.username = "ubuntu"
 override.ssh.private_key_path = "#{YOUR_AWS_PRIVATE_KEY_PATH}"
 override.vm.synced_folder "#{YOUR_SYNC_FOLDER}", "/vagrant", type: "rsync"
 override.vm.box = "dimroc/awsdummy"
 end

end

LOCAL_BOXS = {
 "ubuntu1404x64" => "ubuntu/trusty64",
 "ubuntu1210x64" => "chef/ubuntu-12.10"
}

AWS_AMIS = {
 "ubuntu1404x64" => "ami-864d84ee",
 "ubuntu1210x64" => "ami-02df496b",
 "windows2012r2x64" => "ami-9ade1df2",
 "windows2012x64" => "ami-5ce32034",
 "windows2008r2x64" => "ami-2ae02342",
 "windows2008x64" => "ami-5e24e936",
 "windows2003r2x64" => "ami-b0e320d8"
}

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
 config.vm.box = "precise64_vmware"

 # vmware_fusion
 config.vm.provider "vmware_fusion" do |v, override|
 override.vm.box = "precise64_vmware"
 v.gui = false
 v.vmx["memsize"] = "1024"
 v.vmx["numvcpus"] = "2"
 end

 # vsphere
 config.vm.provider :vsphere do |vsphere|
 vsphere.vm.box = "nkhasanov/vsphere-simple"
 vsphere.host = '#{}'
 vsphere.compute_resource_name = '#{}'
 # vsphere.resource_pool_name = 'YOUR RESOURCE POOL'
 vsphere.insecure = true
 vsphere.template_name = 'qasus-tw-centos7x64-01'
 vsphere.name = "#{YOUR_NAME}-test-machine"
 vsphere.user = 'administrator'
 vsphere.password = '#{}'
 vsphere.vm_base_path = 'vmware_template'
 vsphere.linked_clone = true
 end

 # virtual box
 config.vm.provider "virtualbox" do |vb|
 vb.gui = false
 vb.memory = 1024
 vb.cpus = 2
 end

 # aws
 config.vm.provider :aws do |aws, override|
 # aws configurations
 aws.access_key_id = "#{YOUR_AWS_ACCESS_KEY_ID}"
 aws.secret_access_key = "#{YOUR_AWS_ACCESS_KEY}"
 aws.keypair_name = "#{YOUR_NAME}"
 aws.security_groups = "#{YOUR_NAME}"
 aws.instance_type = "t2.small"
 aws.region = "us-east-1"
 # ubuntu 14.04 x64
 aws.ami = "ami-864d84ee"

 # override info
 override.ssh.username = "ubuntu"
 override.ssh.private_key_path = "#{YOUR_AWS_PRIVATE_KEY_PATH}"
 override.vm.synced_folder "#{YOUR_SYNC_FOLDER}", "/vagrant", type: "rsync"
 override.vm.box = "dimroc/awsdummy"
 end

 # VMs
 (1..MAX_VM_NUMBER).each do |i|
 # define linux
 config.vm.define "l#{i}" do |node|
 # aws
 node.vm.provider :aws do |aws|
 aws.tags = {
 "Name" => "#{YOUR_NAME}-linux-#{i}"
 }
 end
 # local
 # node.vm.network "private_network", ip: "192.168.33.%d" % (i+2)
 node.vm.hostname = "ftan-linux-#{i}"
 end
 # define windows
 config.vm.define "w#{i}" do |node|
 node.vm.provider :aws do |aws|
 aws.ami = "ami-2ae02342"
 aws.tags = {
 "Name" => "#{YOUR_NAME}-windows-#{i}"
 }
 end
 end
 end

 # define customer
 YOUR_CUSTOMIZED_VM.each do |vm|
 config.vm.define "%s" % vm["name"] do |node|
 # aws
 node.vm.provider :aws do |aws|
 aws.ami = AWS_AMIS[vm["platform"]]
 aws.tags = {
 "Name" => "#{YOUR_NAME}-%s" % vm["name"]
 }
 end
 # vmware fusion
 # vmware workstation
 node.vm.provider :vmware_fusion do |fusion|
 fusion.vm.box = LOCAL_BOXS[vm["platform"]]
 fusion.vm.hostname = "#{YOUR_NAME}-%s" % vm["name"]
 end
 # vsphere
 # azure
 end
 end
end

How to configure time zone and NTP on RHEL7/CentOS7

Tue, 29 Sep 2020 11:41:43 +0800

How to configure time zone and NTP on RHEL7/CentOS7

chrony includes two programs: chronyd is a daemon that starts on boot, and chronyc is a command-line client that can monitor chronyd and change runtime parameters.

Use either ntpd or chronyd, not both.

Configure time zone

~# timedatectl set-timezone Asia/Taipei
~# timedatectl
 Local time: Tue 2018-03-27 14:13:38 CST
 Universal time: Tue 2018-03-27 06:13:38 UTC
 RTC time: Tue 2018-03-27 06:13:40
 Time zone: Asia/Taipei (CST, +0800)
 NTP enabled: no
NTP synchronized: no
 RTC in local TZ: no
 DST active: n/a

Configure chronyd

# Install
~# yum install -y chrony

# Config file
~# cat /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.tw.pool.ntp.org iburst --->改成本地的伺服器
server 1.tw.pool.ntp.org iburst --->改成本地的伺服器
server 2.tw.pool.ntp.org iburst --->改成本地的伺服器
server 3.tw.pool.ntp.org iburst --->改成本地的伺服器

# Start service and enable on boot
~# systemctl enable chronyd
~# systemctl start chronyd

tracking parameters show system time performance

~# chronyc tracking
Reference ID : 3DD8996B (61-216-153-107.hinet-ip.hinet.net) --->表示現在同步的時間伺服器，如果沒有id表示沒有同步
Stratum : 4 --->表示計算機有多少"跳hop" 表示本地的是第四層
Ref time (UTC) : Tue Mar 27 06:03:38 2018 --->最後一次測量的時間
System time : 0.000040356 seconds fast of NTP time --->調整系統時間
Last offset : +0.000163738 seconds
RMS offset : 0.000163738 seconds
Frequency : 21.384 ppm fast
Residual freq : +0.000 ppm
Skew : 675.319 ppm
Root delay : 0.008527911 seconds
Root dispersion : 0.066466033 seconds
Update interval : 2.0 seconds
Leap status : Normal --->Normal要顯示此值, Insert second, Delete second or Not synchronised.

~# chronyc sources -v
210 Number of sources = 4

 .-- Source mode '^' = server, '=' = peer, '#' = local clock.
 / .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \  | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 59-124-29-241.hinet-ip.h> 3 6 37 24 -1462us[-2363us] +/- 49ms
^+ 61-216-153-107.hinet-ip.> 3 6 37 23 -556us[ -556us] +/- 64ms
^? 59-125-122-217.hinet-ip.> 0 7 0 - +0ns[ +0ns] +/- 0ns
^- 61-216-153-105.hinet-ip.> 3 6 37 23 -280us[ -280us] +/- 64ms

View sync source info

~# chronyc sourcestats -v
210 Number of sources = 4
 .- Number of sample points in measurement set.
 / .- Number of residual runs with same sign.
 | / .- Length of measurement set (time).
 | | / .- Est. clock freq error (ppm).
 | | | / .- Est. error in freq.
 | | | | / .- Est. offset.
 | | | | | | On the -.
 | | | | | | samples. \
 | | | | | | |
Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev
==============================================================================
59-124-29-241.hinet-ip.h> 6 5 135 -0.454 4.553 -784us 66us
61-216-153-107.hinet-ip.> 6 6 135 +4.455 19.761 +622us 247us
59-125-122-217.hinet-ip.> 0 0 0 +0.000 2000.000 +0ns 4000ms
61-216-153-105.hinet-ip.> 6 4 136 +8.965 42.440 +1250us 495us

Write system time to hardware clock

~# hwclock --systohc
~# date ; hwclock
Tue Mar 27 14:07:57 CST 2018
Tue 27 Mar 2018 02:07:58 PM CST -0.938012 seconds

Ansible Introduction

Sat, 26 Sep 2020 16:57:21 +0800

Getting to know Ansible.

Introduction

安裝部署工具、設定管理工具等
同類型工具：Chef、Puppet、SaltStack
不需要 Agent、透過 ssh
Linux 有 python 即可 ( ssh port )
Win 啟用 winrm 即可 ( 5986 port )
- https://docs.ansible.com/ansible/latest/user_guide/windows_winrm.html#inventory-options
資料夾結構簡單易懂、官方文件豐富易懂、模組多支援設備多、易撰寫

Install

pip install ansible
- pip3 install ansible
yum install ansible
apt-get install ansible
apk add ansible

Common modules

Common modules - ping

Install PowerDNS and PowerDNS-Admin on Ubuntu 22.04|20.04|18.04

Fri, 25 Sep 2020 09:38:17 +0800

Install PowerDNS

$ sudo apt update
$ sudo apt install mariadb-server -y

$ sudo mysql -u root

CREATE DATABASE powerdns;
GRANT ALL ON powerdns.* TO 'powerdns'@'localhost' IDENTIFIED BY 'Str0ngPasswOrd';
FLUSH PRIVILEGES;
USE powerdns;
CREATE TABLE domains (
 id INT AUTO_INCREMENT,
 name VARCHAR(255) NOT NULL,
 master VARCHAR(128) DEFAULT NULL,
 last_check INT DEFAULT NULL,
 type VARCHAR(6) NOT NULL,
 notified_serial INT UNSIGNED DEFAULT NULL,
 account VARCHAR(40) CHARACTER SET 'utf8' DEFAULT NULL,
 PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE UNIQUE INDEX name_index ON domains(name);


CREATE TABLE records (
 id BIGINT AUTO_INCREMENT,
 domain_id INT DEFAULT NULL,
 name VARCHAR(255) DEFAULT NULL,
 type VARCHAR(10) DEFAULT NULL,
 content VARCHAR(64000) DEFAULT NULL,
 ttl INT DEFAULT NULL,
 prio INT DEFAULT NULL,
 change_date INT DEFAULT NULL,
 disabled TINYINT(1) DEFAULT 0,
 ordername VARCHAR(255) BINARY DEFAULT NULL,
 auth TINYINT(1) DEFAULT 1,
 PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE INDEX nametype_index ON records(name,type);
CREATE INDEX domain_id ON records(domain_id);
CREATE INDEX ordername ON records (ordername);


CREATE TABLE supermasters (
 ip VARCHAR(64) NOT NULL,
 nameserver VARCHAR(255) NOT NULL,
 account VARCHAR(40) CHARACTER SET 'utf8' NOT NULL,
 PRIMARY KEY (ip, nameserver)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE TABLE comments (
 id INT AUTO_INCREMENT,
 domain_id INT NOT NULL,
 name VARCHAR(255) NOT NULL,
 type VARCHAR(10) NOT NULL,
 modified_at INT NOT NULL,
 account VARCHAR(40) CHARACTER SET 'utf8' DEFAULT NULL,
 comment TEXT CHARACTER SET 'utf8' NOT NULL,
 PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE INDEX comments_name_type_idx ON comments (name, type);
CREATE INDEX comments_order_idx ON comments (domain_id, modified_at);


CREATE TABLE domainmetadata (
 id INT AUTO_INCREMENT,
 domain_id INT NOT NULL,
 kind VARCHAR(32),
 content TEXT,
 PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE INDEX domainmetadata_idx ON domainmetadata (domain_id, kind);


CREATE TABLE cryptokeys (
 id INT AUTO_INCREMENT,
 domain_id INT NOT NULL,
 flags INT NOT NULL,
 active BOOL,
 content TEXT,
 PRIMARY KEY(id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE INDEX domainidindex ON cryptokeys(domain_id);


CREATE TABLE tsigkeys (
 id INT AUTO_INCREMENT,
 name VARCHAR(255),
 algorithm VARCHAR(50),
 secret VARCHAR(255),
 PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE UNIQUE INDEX namealgoindex ON tsigkeys(name, algorithm);

$ sudo systemctl disable systemd-resolved
$ sudo systemctl stop systemd-resolved

$ ls -lh /etc/resolv.conf
lrwxrwxrwx 1 root root 39 Jul 24 15:50 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf
$ sudo unlink /etc/resolv.conf

$ echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf

Add official PowerDNS repository for Ubuntu 22.04|20.04|18.04.

Set interface IP with netplan on Ubuntu 18.04

Fri, 18 Sep 2020 13:00:05 +0800

Set interface IP with netplan on Ubuntu 18.04

Following the notes above, check /etc/netplan and open /etc/netplan/50-cloud-init.yaml:

# This file is generated from information provided by
# the datasource. Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
 ethernets:
 ens192:
 dhcp4: true
 ens224:
 dhcp4: true
 version: 2

It looks like you can disable cloud network, but I do not use cloud-init, so remove it:

Deploying OpenVPN with AD domain authentication

Thu, 17 Sep 2020 13:15:33 +0800

# Install OpenVPN
yum install openvpn -y
yum -y install openssl openssl-devel -y
yum -y install lzo lzo-devel -y
yum install -y libgcrypt libgpg-error libgcrypt-devel

# Install OpenVPN auth plugin
yum install openvpn-auth-ldap -y

# Install easy-rsa
# Since openvpn 2.3 removed easy-rsa from the package, install it separately.
yum install easy-rsa
cp -rf /usr/share/easy-rsa/2.0 /etc/opevpn/

# Generate OpenVPN keys and certificates
# Edit `/opt/openvpn/etc/easy-rsa/2.0/vars` parameters
export KEY_COUNTRY="CN" # Country
export KEY_PROVINCE="ZJ" # Province
export KEY_CITY="NingBo" # City
export KEY_ORG="TEST-VPN" # Organization
exportKEY_EMAIL="81367070@qq.com" # Email
export KEY_OU="baidu" # Unit

source vars
./clean-all
./build-ca
./build-dh
./build-key-server server
./build-key client1


# Edit the OpenVPN server config: `/etc/openvpn/server.conf`
port 1194
proto udp
dev tun
ca keys/ca.crt
cert keys/server.crt
key keys/server.key # This file should be kept secret
dh keys/dh2048.pem
server 10.8.0.0 255.255.255.0 // client IP pool
push "route 192.168.1.0 255.255.255.0" // push route to clients
push "redirect-gateway" // change client gateway to route VPN traffic
ifconfig-pool-persist ipp.txt
keepalive 10 120
comp-lzo
persist-key
persist-tun
status openvpn-status.log
verb 3
plugin /usr/lib64/openvpn/plugin/lib/openvpn-auth-ldap.so "/etc/openvpn/auth/ldap.conf"
client-cert-not-required
username-as-common-name
log /var/log/openvpn.log


# Edit openvpn-ldap-auth config: `/etc/openvpn/auth/ldap.conf`
# /etc/openvpn/auth/ldap.conf

<LDAP>
 # LDAP server URL
 # Change to the AD server IP
 URL ldap://172.16.76.238:389

 # Bind DN (If your LDAP server doesn't support anonymous binds)
 # BindDN uid=Manager,ou=People,dc=example,dc=com
 # Change to the domain admin DN; you can query it with ldapsearch
 # Replace the IP in -h with the server IP, -D with the admin DN, -b with the base DN, and * for all
 # ldapsearch -LLL -x -h 172.16.76.238 -D "administrator@xx.com" -W -b "dc=xx,dc=com" "*"
 BindDN "cn=administrator,cn=Users,dc=xx,dc=com"

 # Bind Password
 # Password SecretPassword
 # Domain admin password
 Password passwd

 # Network timeout (in seconds)
 Timeout 15

 # Enable Start TLS
 TLSEnable no

 # Follow LDAP Referrals (anonymously)
 FollowReferrals no

 # TLS CA Certificate File
 # TLSCACertFile /usr/local/etc/ssl/ca.pem

 # TLS CA Certificate Directory
 # TLSCACertDir /etc/ssl/certs

 # Client Certificate and key
 # If TLS client authentication is required
 # TLSCertFile /usr/local/etc/ssl/client-cert.pem
 # TLSKeyFile /usr/local/etc/ssl/client-key.pem

 # Cipher Suite
 # The defaults are usually fine here
 # TLSCipherSuite ALL:!ADH:@STRENGTH
</LDAP>

<Authorization>
 # Base DN
 # Base DN for auth search
 BaseDN "dc=boqii-inc,dc=com"

 # User Search Filter
 # SearchFilter "(&(uid=%u)(accountStatus=active))"
 # sAMAccountName=%u uses the sAMAccountName value as the username,
 # and "memberof=CN=myvpn,DC=xx,DC=com" points to the VPN user group to authenticate,
 # so any user can use VPN once they are in this group.
 SearchFilter "(&(sAMAccountName=%u)(memberof=CN=myvpn,DC=boqii-inc,DC=com))"

 # Require Group Membership
 RequireGroup false

 # Add non-group members to a PF table (disabled)
 # PFTable ips_vpn_users

 <Group>
 # BaseDN "ou=Groups,dc=example,dc=com"
 # SearchFilter "(|(cn=developers)(cn=artists))"
 # MemberAttribute uniqueMember
 # Add group members to a PF table (disabled)
 # PFTable ips_vpn_eng

 BaseDN "ou=vpn,dc=boqii-inc,dc=com"
 SearchFilter "(cn=openvpn)"
 MemberAttribute "member"
 </Group>
</Authorization>

Copy the ca.crt certificate under /etc/openvpn/key for client use.

Docker Tips: Using Docker Config

Mon, 14 Sep 2020 11:13:23 +0800

Docker Tips: Using Docker Config

FROM nginx:1.13.6
COPY nginx.conf /etc/nginx/nginx.conf

Using the Docker CLI, we can create a config from this configuration file, we name this config proxy.

$ docker config create proxy nginx.conf
mdcfnxud53ve6jgcgjkhflg0s

$ docker config inspect proxy
[
 {
 "ID": "x06uaozphg9kbnf8g4az4mucn",
 "Version": {
 "Index": 2723
 },
 "CreatedAt": "2017-11-21T07:49:09.553666064Z",
 "UpdatedAt": "2017-11-21T07:49:09.553666064Z",
 "Spec": {
 "Name": "proxy",
 "Labels": {},
 "Data": "dXNlciB3d3ctZGF0YTsKd29y...ogIgICAgIH0KICAgIH0KfQo="
 }
 }
]

Use a Config

$ docker network create --driver overlay front
$ docker service create --name api --network front lucj/api
$ docker service create --name proxy \
 --network front \
 --config src=proxy,target=/etc/nginx/nginx.conf \
 --port 8000:8000 \
 nginx:1.13.6

Service Update

When the content of a configuration needs to be modified, it’s a common pattern to create a new config (using docker config create), and then to update the service order to remove the access to the previous config, and to add the access to the new one. The service commands are--config-rm and --config-add.

Fortigate Management Interface in HA Mode

Tue, 08 Sep 2020 09:47:47 +0800

Fortigate Management Interface in HA Mode

First you activate the feature

config system ha
set ha-mgmt-status enable
config ha-mgmt-interfaces
edit 1
set interface wan2
set gateway 192.168.147.254
next
end
end

Do not forget to set a default gateway. This interface is isolated and requires its own routing.

Then you assign an individual IP address to every node in the cluster

System1

config system interface
edit wan2
set ip 10.11.101.101/24
set allowaccess https ping ssh snmp
next
end

System2

How to automatically resize virtual box disk with vagrant

Tue, 25 Aug 2020 09:40:34 +0800

Vagrant.configure(2) do |config|
config.vm.box = "centos/7"
config.disksize.size = '20GB'
end

$ sudo parted /dev/sda resizepart 2 100%

$ sudo lvextend -l +100%FREE /dev/centos/root

$ sudo xfs_growfs /dev/centos/root

Automate Part

Vagrant.configure(2) do |config|
common = <<-SCRIPT
sudo parted /dev/sda resizepart 2 100%
sudo pvresize /dev/sda2
sudo lvextend -l +100%FREE /dev/centos/root
sudo xfs_growfs /dev/centos/root
SCRIPT
config.vm.define "node01" do |node1|
node1.vm.hostname = "node01"
node1.vm.network "private_network", ip: "192.168.56.121"
config.vm.provision :shell, :inline => common
end
end

vagrant plugin install vagrant-disksize

Vagrantfile

# Fail if the vagrant-disksize plugin is not installed
unless Vagrant.has_plugin?("vagrant-disksize")
 raise 'vagrant-disksize is not installed!'
end
Vagrant.configure("2") do |config|
 config.vm.provider "virtualbox" do |vb|
 vb.name = "DISKEXTEND"
 vb.memory = 2048
 vb.cpus = 2
 end
 config.vm.define :"DISKEXTEND" do |t|
 end
 config.vm.hostname = "DISKEXTEND"
 config.vm.box = "bento/ubuntu-18.04"
 # Increase the default disk size of the bento image (64GB) to 96GB
 config.disksize.size = "96GB"
 # Run a script on provisioning the box to format the file system
 config.vm.provision "shell", path: "disk-extend.sh"
end

Provisioning Script: disk-extend.sh

#!/bin/bash
echo "> Installing required tools for file system management"
if [ -n "$(command -v yum)" ]; then
 echo ">> Detected yum-based Linux"
 sudo yum makecache
 sudo yum install -y util-linux
 sudo yum install -y lvm2
 sudo yum install -y e2fsprogs
fi
if [ -n "$(command -v apt-get)" ]; then
 echo ">> Detected apt-based Linux"
 sudo apt-get update -y
 sudo apt-get install -y fdisk
 sudo apt-get install -y lvm2
 sudo apt-get install -y e2fsprogs
fi
ROOT_DISK_DEVICE="/dev/sda"
ROOT_DISK_DEVICE_PART="/dev/sda1"
LV_PATH=`sudo lvdisplay -c | sed -n 1p | awk -F ":" '{print $1;}'`
FS_PATH=`df / | sed -n 2p | awk '{print $1;}'`
ROOT_FS_SIZE=`df -h / | sed -n 2p | awk '{print $2;}'`
echo "The root file system (/) has a size of $ROOT_FS_SIZE"
echo "> Increasing disk size of $ROOT_DISK_DEVICE to available maximum"
sudo fdisk $ROOT_DISK_DEVICE <<EOF
d
n
p
1
2048
no
w
EOF
sudo pvresize $ROOT_DISK_DEVICE_PART
sudo lvextend -l +100%FREE $LV_PATH
sudo resize2fs -p $FS_PATH
ROOT_FS_SIZE=`df -h / | sed -n 2p | awk '{print $2;}'`
echo "The root file system (/) has a size of $ROOT_FS_SIZE"
exit 0

Common GitBook plugins

Tue, 18 Aug 2020 09:48:41 +0800

Common GitBook plugins

Google Search Operators: The Complete List (44 Advanced Operators)

Fri, 14 Aug 2020 14:21:06 +0800

Google Search Operators: The Complete List (44 Advanced Operators)

Working

Search operator	What it does	Example
`" "`	Search for results that mention a word or phrase.	`"steve jobs"`
`OR`	Search for results related to X or Y.	`jobs OR gates`
`\|`	Same as `OR`.	`jobs \| gates`
`AND`	Search for results related to X and Y.	`jobs AND gates`
`-`	Search for results that don’t mention a word or phrase.	`jobs -apple`
`*`	Wildcard matching any word or phrase.	`steve * apple`
`( )`	Group multiple searches.	`(ipad OR iphone) apple`
`define:`	Search for the definition of a word or phrase.	`define:entrepreneur`
`cache:`	Find the most recent cache of a webpage.	`cache:apple.com`
`filetype:`	Search for particular types of files (e.g., PDF).	`apple filetype:pdf`
`ext:`	Same as `filetype:`	`apple ext:pdf`
`site:`	Search for results from a particular website.	`site:apple.com`
`related:`	Search for sites related to a given domain.	`related:apple.com`
`intitle:`	Search for pages with a particular word in the title tag.	`intitle:apple`
`allintitle:`	Search for pages with multiple words in the title tag.	`allintitle:apple iphone`
`inurl:`	Search for pages with a particular word in the URL.	`inurl:apple`
`allinurl:`	Search for pages with multiple words in the URL.	`allinurl:apple iphone`
`intext:`	Search for pages with a particular word in their content.	`intext:apple iphone`
`allintext:`	Search for pages with multiple words in their content.	`allintext:apple iphone`
`weather:`	Search for the weather in a location.	`weather:san francisco`
`stocks:`	Search for stock information for a ticker.	`stocks:aapl`
`map:`	Force Google to show map results.	`map:silicon valley`
`movie:`	Search for information about a movie.	`movie:steve jobs`
`in`	Convert one unit to another.	`$329 in GBP`
`source:`	Search for results from a particular source in Google News.	`apple source:the_verge`
`before:`	Search for results from before a particular date.	`apple before:2007-06-29`
`after:`	Search for results from after a particular date.	`apple after:2007-06-29`

Unreliable

Search operator	What it does	Example
`#..#`	Search within a range of numbers.	`iphone case $50..$60`
`inanchor:`	Search for pages with backlinks containing specific anchor text.	`inanchor:apple`
`allinanchor:`	Search for pages with backlinks containing multiple words in their anchor text.	`allinanchor:apple iphone`
`AROUND(X)`	Search for pages with two words or phrases within X words of one another.	`apple AROUND(4) iphone`
`loc:`	Find results from a given area.	`loc:"san francisco" apple`
`location:`	Find news from a certain location in Google News.	`location:"san francisco" apple`
`daterange:`	Search for results from a particular date range.	`daterange:11278-13278`

Not working (officially dropped by Google)

Search operator	What it does	Example
`~`	Include synonyms in the search (dropped 2013).	`~apple`
`+`	Search for results mentioning an exact word or phrase (dropped 2011).	`jobs +apple`
`inpostauthor:`	Search for posts by a specific author in Google Blog Search (discontinued).	`inpostauthor:"steve jobs"`
`allinpostauthor:`	Same as `inpostauthor:`, but removes the need for quotes.	`allinpostauthor:steve jobs`
`inposttitle:`	Search for posts with certain words in the title in Google’s discontinued Blog Search.	`inposttitle:apple iphone`
`link:`	Search for pages linking to a particular domain or URL (dropped 2017).	`link:apple.com`
`info:`	Search for information about a specific page or website (dropped 2017).	`info:apple.com`
`id:`	Same as `info:`	`id:apple.com`
`phonebook:`	Search for someone’s phone number (dropped 2010).	`phonebook:tim cook`
`#`	Search for hashtags on Google+ (dropped 2019).	`#apple`

An easy way to identify virtualization technology

Wed, 29 Jul 2020 21:11:45 +0800

An easy way to identify virtualization technology

`dmidecode -s system-product-name`

Virtualization technology

VMware Workstation

root@router:~# dmidecode -s system-product-name
VMware Virtual Platform

VirtualBox

root@router:~# dmidecode -s system-product-name
VirtualBox

QEMU and KVM

root@router:~# dmidecode -s system-product-name
KVM

QEMU (emulation)

root@router:~# dmidecode -s system-product-name
Bochs

Microsoft Virtual PC

root@router:~# dmidecode | egrep -i 'manufacturer|product'
Manufacturer: Microsoft Corporation
Product Name: Virtual Machine

Virtuozzo

root@router:~# dmidecode
/dev/mem: Permission denied

en

root@router:~# dmidecode | grep -i domU
Product Name: HVM domU

`/dev/disk/by-id`

If you do not have permission to run dmidecode, you can use: ls -1 /dev/disk/by-id/

[Share] Keychain - Pendant from a Type 57 rifle bayonet scabbard

Fri, 10 Jul 2020 12:32:27 +0800

[Share] Keychain - Pendant from a Type 57 rifle bayonet scabbard

SSH certificate login guide

Wed, 08 Jul 2020 13:39:48 +0800

SSH certificate login guide

Before using SSH certificate login, you need to generate certificates. The steps are:

The user and the server both send their public keys to the CA.
The CA uses the server public key to generate a server certificate and sends it to the server.
The CA uses the user public key to generate a user certificate and sends it to the user.

Once certificates are in place, the user can log in. SSH handles the whole process automatically.

How to capture web pages and long screenshots with Chrome DevTools?

Thu, 02 Jul 2020 09:07:34 +0800

How to capture web pages and long screenshots with Chrome DevTools?

Open Chrome DevTools
Click more_vert in the top-right, then choose filter_none to make it a separate window
Ctrl + Shift + P(Command + Shift + P)
Type screenshot
1. Capture area screenshot: area screenshot. After selecting it, your cursor changes to add. Use it to select an area on the page.
2. Capture full size screenshot: long screenshot. It completes automatically after about one second.
3. Capture node screenshot: node screenshot. Similar to a long screenshot, but only visible parts are captured and the rest is blank.
4. Capture screenshot: normal screenshot. It captures the visible page. If DevTools is not in a separate window, the result is affected.

BIRD and BGP: a beginner's kickoff

Mon, 22 Jun 2020 09:38:55 +0800

Introduction to Git internals

Tue, 02 Jun 2020 08:39:08 +0800

Introduction to Git internals

Using Python for multimedia: video, audio, and images

Sun, 31 May 2020 17:48:10 +0800

Using Python for multimedia: video, audio, and images

Aliyun CDN Cache Rules

Tue, 12 May 2020 21:51:55 +0800

Abount Ansible Hosts

Tue, 12 May 2020 19:46:04 +0800

# hosts 可以這樣寫

$ ansible 'max-compute[!3:9][0:3][0:3]' -m setup --list-hosts
 hosts (4):
 max-compute10
 max-compute13
 max-compute20
 max-compute23
$ ansible 'max-compute[!3:9][0:4][0:4]' -m setup --list-hosts
 hosts (3):
 max-compute10
 max-compute14
 max-compute20
$ ansible 'max-compute[!0:3]' -m setup --list-hosts
 hosts (6):
 max-compute4
 max-compute5
 max-compute6
 max-compute7
 max-compute8
 max-compute9

plugin: gcp_compute
projects:
 - project-XXX
zones:
 - asia-east1-a
 - asia-east1-c
 - asia-east1-b
 - asia-east2-a
 - asia-east2-c
 - asia-east2-b
auth_kind: serviceaccount
service_account_file: /root/.ssh/sa001.json
filters:
 - status="RUNNING"
hostnames:
 - name
groups:
 nginx: "'nginx' in name"
 cassandra: "'cassandra' in name"
 redis: "'redis' in name"
 misc: "name in ('noc', 'mt-center')"
 prod: "'prod' in name and 'gke' not in name"
 all: "'gke' not in name"
 search: "'search' in name"

cht_domain:
 "{{ 'web.qat.cht' if ansible_default_ipv4['network'] == '192.168.40.0'
 else 'ap.qat.cht' if ansible_default_ipv4['network'] == '192.168.50.0'
 else 'db.qat.cht' if ansible_default_ipv4['network'] == '192.168.60.0'
 else 'qat.cht' }}"

nginx 添加第三方nginx_upstream_check_module 模块实现健康状态检测

Sun, 26 Apr 2020 20:05:37 +0800

nginx.conf

 http {
 upstream cluster {
 # simple round-robin
 server 192.168.0.1:80;
 server 192.168.0.2:80;

 check interval=5000 rise=1 fall=3 timeout=4000;
 #check interval=3000 rise=2 fall=5 timeout=1000 type=ssl_hello;
 #check interval=3000 rise=2 fall=5 timeout=1000 type=http;
 #check_http_send "HEAD / HTTP/1.0\r\n\r\n";
 #check_http_expect_alive http_2xx http_3xx;
 }
...

 check
 syntax: *check interval=milliseconds [fall=count] [rise=count]
 [timeout=milliseconds] [default_down=true|false]
 [type=tcp|http|ssl_hello|mysql|ajp|fastcgi]*

 默认配置：interval=3000 fall=5 rise=2 timeout=1000 default_down=true type=tcp*
...

interval：检测间隔 3 秒
fall: 连续检测失败次数 5 次时，认定 relaserver is down
rise: 连续检测成功 2 次时，认定 relaserver is up
timeout: 超时 1 秒
default_down: 初始状态为 down,只有检测通过后才为 up
type: 检测类型方式 tcp
1. tcp :tcp 套接字,不建议使用，后端业务未 100%启动完成,前端已经放开访问的情况
2. ssl_hello：发送 hello 报文并接收 relaserver 返回的 hello 报文
3. http: 自定义发送一个请求，判断上游 relaserver 接收并处理
4. mysql: 连接到 mysql 服务器，判断上游 relaserver 是否还存在
5. ajp: 发送 AJP Cping 数据包，接收并解析 AJP Cpong 响应以诊断上游 relaserver 是否还存活(AJP tomcat 内置的一种协议)
6. fastcgi: php 程序是否存活

example

CS Visualized: Useful Git Commands

Thu, 16 Apr 2020 20:20:16 +0800

CS Visualized: Useful Git Commands

string field was converted to True (type string)

Wed, 15 Apr 2020 21:34:09 +0800

Surprisingly, you can view images in the Linux CLI?

Tue, 14 Apr 2020 22:01:02 +0800

Surprisingly, you can view images in the Linux CLI?

FIM

sudo apt-get install fim

Common shortcuts for images in FIM:

PageUp / Down: previous/next image
+/-: zoom in/out
a: auto scale
w: fit width
h: fit height
j / k: pan down/up
f / m: flip/mirror
r / R: rotate (clockwise/counterclockwise)
ESC / q: quit

Viu

cargo install viu

Lsix

sudo apt-get install imagemagick

wget https://github.com/hackerb9/lsix/archive/master.zip

How to deploy on remote Docker hosts with docker-compose

Wed, 25 Mar 2020 19:30:54 +0800

How to deploy on remote Docker hosts with docker-compose

Manual deployment by copying project files, install docker-compose and running it

$ scp -r hello-docker user@remotehost:/path/to/src
$ ssh user@remotehost
$ pip install docker-compose
$ cd /path/to/src/hello-docker
$ docker-compose up -d

Using DOCKER_HOST environment variable to set up the target engine

$ cd hello-docker
$ DOCKER_HOST="ssh://user@remotehost" docker-compose up -d

Using docker contexts

$ docker context create remote ‐‐docker "host=ssh://user@remotemachine"
remote
Successfully created context "remote"

$ docker context ls
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * Current DOCKER_HOST… unix:///var/run/docker.sock swarm
remote ssh://user@remotemachine

$ cd hello-docker
$ docker-compose ‐‐context remote up -d

Basic kubeadm usage notes (by request)

Tue, 24 Mar 2020 16:00:34 +0800

Vim Tips - Edit Remote Files With Vim On Linux

Sat, 14 Mar 2020 15:43:38 +0800

Vim Tips - Edit Remote Files With Vim On Linux

Edit remote files with Vim on Linux

$ vim scp://sk@192.168.225.22/info.txt

user@remotesystem:port (E.g. sk@192.168.225.22)
Single slash (/) - If you want to edit a file that is stored in the $HOME directory of a remote system, you must use a trailing slash to separate remote system’s IP address or hostname from the file path.
Double slashes (//) - To specify full path of a file, you must use double slashes. For example, let us say you are editing a file named info.txt that is located in /home/sk/Documents/ directory of your remote system. In this case, the command would be: vim scp://sk@192.168.225.22//home/sk/Documents/info.txt

Python install module issues

Wed, 04 Mar 2020 15:43:38 +0800

I can’t install python-ldap

python-ldap

https://www.python-ldap.org/en/latest/installing.html

$ pip install python-ldap
In file included from Modules/LDAPObject.c:9:


Modules/errors.h:8: fatal error: lber.h: No such file or directory

Debian/Ubuntu: sudo apt-get install libsasl2-dev python-dev libldap2-dev libssl-dev
RedHat/CentOS:

sudo yum install python-devel openldap-devel
sudo yum groupinstall "Development tools"

molecule

$ pip install molecule
error: command 'gcc' failed with exit status 1


 ----------------------------------------
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-I5DGC3/psutil/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-Fy7V4X/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-I5DGC3/psutil/

RedHat/CentOS:

yum -y install gcc gcc-c++ kernel-devel
yum -y install python-devel libxslt-devel libffi-devel openssl-devel

ansible

$ pip install ansible
ImportError: No module named pkg_resources

yum install -y python-setuptools

`import pandas`

ModuleNotFoundError: No module named '_bz2'

apt-get install -y libbz2-dev
yum install -y bzip2-devel

`from .cv2 import *`

ImportError: libSM.so.6: cannot open shared object file: No such file or directory
ImportError: libXrender.so.1: cannot open shared object file: No such file or directory
ImportError: libXext.so.6: cannot open shared object file: No such file or directory

Ubuntu:

apt-get install libsm6
apt-get install libxrender1
apt-get install libxext-dev

CentOS:

yum install libSM
yum install libXrender-devel
yum install libXext

vagrant筆記

Wed, 26 Feb 2020 10:52:19 +0800

1 General 1.1 Path 下載的 box 預設放在~/.vagrant.d/boxes 裡 guest machine 的 data 放的地方根據 VM 設定而不同可開啟 VirtualBox 查看，VirtualBox 預設是在~/VirtualBox\ VMS

1.2 Shared Folder guest machine 的/vagrant 將會自動 mount 到 host machine 的放 Vagrantfile 的那個 folder 此功能需要 guest machine 的 Box 的 VB guest additions 版本與 VM 中的一樣若出現錯誤則需要更新 box 或用這個非官方的 plugin: https://github.com/dotless-de/vagrant-vbguest

2 Uninstall 2.1 REMOVING THE VAGRANT PROGRAM

rm -rf /Applications/Vagrant
rm -f /usr/bin/vagrant
sudo pkgutil --forget com.vagrant.vagrant

2.2 REMOVING USER DATA rm -rf ~/.vagrant.d 3 Command https://www.vagrantup.com/docs/cli/

Percona config

Fri, 07 Feb 2020 21:13:57 +0800

# Percona Server template configuration


[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
port=3306
datadir=/data/mysql
socket=/data/mysql/mysql.sock
pid_file=/data/mysql/mysqld.pid


# 服务端编码
character_set_server=utf8mb4
# 服务端排序
collation_server=utf8mb4_general_ci
# 强制使用 utf8mb4 编码集，忽略客户端设置
skip_character_set_client_handshake=1
# 日志输出到文件
log_output=FILE
# 开启常规日志输出
general_log=1
# 常规日志输出文件位置
general_log_file=/var/log/mysql/mysqld.log
# 错误日志位置
log_error=/var/log/mysql/mysqld-error.log
# 记录慢查询
slow_query_log=1
# 慢查询时间(大于 1s 被视为慢查询)
long_query_time=1
# 慢查询日志文件位置
slow_query_log_file=/var/log/mysql/mysqld-slow.log
# 临时文件位置
tmpdir=/data/mysql_tmp
# 线程池缓存(refs https://my.oschina.net/realfighter/blog/363853)
thread_cache_size=30
# The number of open tables for all threads.(refs https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_table_open_cache)
table_open_cache=16384
# 文件描述符(此处修改不生效，请修改 systemd service 配置)
# refs https://www.percona.com/blog/2017/10/12/open_files_limit-mystery/
# refs https://www.cnblogs.com/wxxjianchi/p/10370419.html
#open_files_limit=65535
# 表定义缓存(5.7 以后自动调整)
# refs https://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_table_definition_cache
# refs http://mysql.taobao.org/monthly/2015/08/10/
#table_definition_cache=16384
sort_buffer_size=1M
join_buffer_size=1M
# MyiSAM 引擎专用(内部临时磁盘表可能会用)
read_buffer_size=1M
read_rnd_buffer_size=1M
# MyiSAM 引擎专用(内部临时磁盘表可能会用)
key_buffer_size=32M
# MyiSAM 引擎专用(内部临时磁盘表可能会用)
bulk_insert_buffer_size=16M
# myisam_sort_buffer_size 与 sort_buffer_size 区别请参考(https://stackoverflow.com/questions/7871027/myisam-sort-buffer-size-vs-sort-buffer-size)
myisam_sort_buffer_size=64M
# 内部内存临时表大小
tmp_table_size=32M
# 用户创建的 MEMORY 表最大大小(tmp_table_size 受此值影响)
max_heap_table_size=32M
# 开启查询缓存
query_cache_type=1
# 查询缓存大小
query_cache_size=32M
# sql mode
sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'


########### Network ###########
# 最大连接数(该参数受到最大文件描述符影响，如果不生效请检查最大文件描述符设置)
# refs https://stackoverflow.com/questions/39976756/the-max-connections-in-mysql-5-7
max_connections=1500
# mysql 堆栈内暂存的链接数量
# 当短时间内链接数量超过 max_connections 时，部分链接会存储在堆栈内，存储数量受此参数控制
back_log=256
# 最大链接错误，针对于 client 主机，超过此数量的链接错误将会导致 mysql server 针对此主机执行锁定(禁止链接 ERROR 1129 )
# 此错误计数仅在 mysql 链接握手失败才会计算，一般出现问题时都是网络故障
# refs https://www.cnblogs.com/kerrycode/p/8405862.html
max_connect_errors=100000
# mysql server 允许的最大数据包大小
max_allowed_packet=64M
# 交互式客户端链接超时(30分钟自动断开)
interactive_timeout=1800
# 非交互式链接超时时间(10分钟)
# 如果客户端有连接池，则需要协商此参数(refs https://database.51cto.com/art/201909/603519.htm)
wait_timeout=600
# 跳过外部文件系统锁定
# If you run multiple servers that use the same database directory (not recommended),
# each server must have external locking enabled.
# refs https://dev.mysql.com/doc/refman/5.7/en/external-locking.html
skip_external_locking=1
# 跳过链接的域名解析(开启此选项后 mysql 用户授权的 host 方式失效)
skip_name_resolve=0
# 禁用主机名缓存，每次都会走 DNS
host_cache_size=0

ini




########### REPL ###########
# 开启 binlog
log_bin=mysql-bin
# 作为从库时，同步信息依然写入 binlog，方便此从库再作为其他从库的主库
log_slave_updates=1
# server id，默认为 ipv4 地址去除第一段
# eg: 172.16.10.11 => 161011
server_id=161011
# 每次次事务 binlog 刷新到磁盘
# refs http://liyangliang.me/posts/2014/03/innodb_flush_log_at_trx_commit-and-sync_binlog/
sync_binlog=100
# binlog 格式(refs https://zhuanlan.zhihu.com/p/33504555)
binlog_format=row
# binlog 自动清理时间
expire_logs_days=10
# 开启 relay-log，一般作为 slave 时开启
relay_log=mysql-replay
# 主从复制时跳过 test 库
replicate_ignore_db=test
# 每个 session binlog 缓存
binlog_cache_size=4M
# binlog 滚动大小
max_binlog_size=1024M
# GTID 相关(refs https://keithlan.github.io/2016/06/23/gtid/)
#gtid_mode=1
#enforce_gtid_consistency=1


########### InnoDB ###########
# 永久表默认存储引擎
default_storage_engine=InnoDB
# 系统表空间数据文件大小(初始化为 1G，并且自动增长)
innodb_data_file_path=ibdata1:1G:autoextend
# InnoDB 缓存池大小
# innodb_buffer_pool_size 必须等于 innodb_buffer_pool_chunk_size*innodb_buffer_pool_instances，或者是其整数倍
# refs https://dev.mysql.com/doc/refman/5.7/en/innodb-buffer-pool-resize.html
# refs https://zhuanlan.zhihu.com/p/60089484
innodb_buffer_pool_size=7680M
innodb_buffer_pool_instances=10
innodb_buffer_pool_chunk_size=128M
# InnoDB 强制恢复(refs https://www.askmaclean.com/archives/mysql-innodb-innodb_force_recovery.html)
innodb_force_recovery=0
# InnoDB buffer 预热(refs http://www.dbhelp.net/2017/01/12/mysql-innodb-buffer-pool-warmup.html)
innodb_buffer_pool_dump_at_shutdown=1
innodb_buffer_pool_load_at_startup=1
# InnoDB 日志组中的日志文件数
innodb_log_files_in_group=2
# InnoDB redo 日志大小
# refs https://www.percona.com/blog/2017/10/18/chose-mysql-innodb_log_file_size/
innodb_log_file_size=256MB
# 缓存还未提交的事务的缓冲区大小
innodb_log_buffer_size=16M
# InnoDB 在事务提交后的日志写入频率
# refs http://liyangliang.me/posts/2014/03/innodb_flush_log_at_trx_commit-and-sync_binlog/
innodb_flush_log_at_trx_commit=2
# InnoDB DML 操作行级锁等待时间
# 超时返回 ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
# refs https://ningyu1.github.io/site/post/75-mysql-lock-wait-timeout-exceeded/
innodb_lock_wait_timeout=30
# InnoDB 行级锁超时是否回滚整个事务，默认为 OFF 仅回滚上一条语句
# 此时应用程序可以接受到错误后选择是否继续提交事务(并没有违反 ACID 原子性)
# refs https://www.cnblogs.com/hustcat/archive/2012/11/18/2775487.html
#innodb_rollback_on_timeout=ON
# InnoDB 数据写入磁盘的方式，具体见博客文章
# refs https://www.cnblogs.com/gomysql/p/3595806.html
innodb_flush_method=O_DIRECT
# InnoDB 缓冲池脏页刷新百分比
# refs https://dbarobin.com/2015/08/29/mysql-optimization-under-ssd
innodb_max_dirty_pages_pct=50
# InnoDB 每秒执行的写IO量
# refs https://www.centos.bz/2016/11/mysql-performance-tuning-15-config-item/#10.INNODB_IO_CAPACITY,%20INNODB_IO_CAPACITY_MAX
innodb_io_capacity=500
innodb_io_capacity_max=1000
# 请求并发 InnoDB 线程数
# refs https://www.cnblogs.com/xinysu/p/6439715.html#_lab2_1_0
innodb_thread_concurrency=60
# 再使用多个 InnoDB 表空间时，允许打开的最大 ".ibd" 文件个数，不设置默认 300，
# 并且取与 table_open_cache 相比较大的一个，此选项独立于 open_files_limit
# refs https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_open_files
innodb_open_files=65535
# 每个 InnoDB 表都存储在独立的表空间(.ibd)中
innodb_file_per_table=1
# 事务级别(可重复读，会出幻读)
transaction_isolation=REPEATABLE-READ
# 是否在搜索和索引扫描中使用间隙锁(gap locking)，不建议使用未来将删除
innodb_locks_unsafe_for_binlog=0
# InnoDB 后台清理线程数，更大的值有助于 DML 执行性能，>= 5.7.8 默认为 4
innodb_purge_threads=4

go-mysql-elasticsearch-benchmarking

Mon, 20 Jan 2020 09:18:00 +0800

go-mysql-elasticsearch

Page: /

Cannot set command timeout per task with network_cli

Tue, 14 Jan 2020 16:34:03 +0800

- name: run command
 ios_command:
 commands:
 - show version
 vars:
 ansible_command_timeout: 40