2023 on Ricky

A python script that allows your terminal to snow.

Sun, 24 Dec 2023 15:40:00 +0800

A python script that allows your terminal to snow.

docker run --rm -ti sontek/snowmachine tree --color rainbow --particle "*" --snow true --lights-color rainbow

Add SFTP user and share directory

Thu, 30 Nov 2023 17:22:00 +0800

dev_test_user, qa_test_user 同權限 dev_user, qa_user 同權限

1. 建立共享資料夾(SFTP 使用的資料夾)

sudo mkdir -p /home/{test,prod}/{exchange,upload}
sudo mkdir -p /home/{test,prod}/exchange/success
sudo mkdir -p /home/{test,prod}/upload/backup

2. 建立使用者群組

sudo groupadd share01-test
sudo groupadd share01-prod

3. 創建 qa_test_user 使用者並設定 qa_test_user 使用者的群組為 share01-test

sudo useradd -m -G share01-test qa_test_user

# 設定 dev_test_user 使用者的群組為 share01-test
sudo usermod -G share01-test dev_test_user

# 設定密碼
sudo passwd qa_test_user

4. 創建 qa_user 使用者並設定 qa_user 使用者的群組為 share01-prod

sudo useradd -m -G share01-prod qa_user

# 設定 dev_user 使用者的群組為 share01-prod
sudo usermod -G share01-prod dev_user

# 設定密碼
sudo passwd qa_user

5. 設定權限

# 設定 /home/test 資料夾(含下級資料夾)的使用者為 qa_test_user，群組為 share01-test
sudo chown -R qa_test_user:share01-test test/

# 設定 /home/prod 資料夾(含下級資料夾)的使用者為 qa_user，群組為 share01-prod
sudo chown -R qa_user:share01-prod prod/

# SFTP 登入資料夾權限要給 root
sudo chown root:root /home/test
sudo chown root:root /home/prod

6. 設定 /etc/ssh/sshd_config

/etc/ssh/sshd_config

Use terminal and SSH to remote host

Fri, 24 Nov 2023 22:22:00 +0800

Use terminal and SSH to remote host

1. Modern Terminals

2. Open terminal in macOS

⌘ + space open Spotlight
search terminal.app
press ↩

3. SSH to remote host

ensure the private key file path.
enter the command in the terminal: ssh -i /path/to/private_key.pem ubuntu@ubuntu.host.com

Fetch GraphQL schema information

Wed, 18 Oct 2023 20:56:00 +0800

curl \
 -XPOST \
 -H "Content-Type: application/json" \
 -d '{"query":"{__schema { types { name enumValues { name } fields { name type {name kind enumValues { name } ofType { name kind ofType {name kind}}}}}}}"}' \
 https://example.com/graphql

Container security fundamentals

Wed, 04 Oct 2023 09:06:00 +0800

Tuning EMQX to Scale to One Million Concurrent Connection on Kubernetes

Wed, 27 Sep 2023 10:36:00 +0800

Linux Kernel Tuning

node level, basically the non-namespaced sysctls

# Sets the maximum number of file handles allowed by the kernel
sysctl -w fs.file-max=2097152

# Sets the maximum number of open file descriptors that a process can have
sysctl -w fs.nr_open=2097152

namespaced sysctls

# Sets the maximum number of connections that can be queued for acceptance by the kernel.
sysctl -w net.core.somaxconn=32768

# Sets the maximum number of SYN requests that can be queued by the kernel
sysctl -w net.ipv4.tcp_max_syn_backlog=16384

# Setting the minimum, default and maximum size of TCP Buffer
sysctl -w net.ipv4.tcp_rmem='1024 4096 16777216'
sysctl -w net.ipv4.tcp_wmem='1024 4096 16777216'

# Setting Parameters for TCP Connection Tracking
sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=30

# Controls the maximum number of entries in the TCP time-wait bucket table
sysctl -w net.ipv4.tcp_max_tw_buckets=1048576

# Controls Timeout for FIN-WAIT-2 Sockets:
sysctl -w net.ipv4.tcp_fin_timeout=15

There are some more namespaced sysctls that will improve the performance but because of an active issue we are not able to set them on the container level

# Sets the size of the backlog queue for the network device
sysctl -w net.core.netdev_max_backlog=16384

# Amount of memory that is allocated for storing incoming and outgoing data for a socket
sysctl -w net.core.rmem_default=262144
sysctl -w net.core.wmem_default=262144

# Setting the maximum amount of memory for the socket buffers
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.core.optmem_max=16777216

Erlang VM Tuning

## Erlang Process Limit
node.process_limit = 2097152

## Sets the maximum number of simultaneously existing ports for this system
node.max_ports = 2097152

EMQX Broker Tuning

# Other configuration…
EMQX_LISTENER__TCP__EXTERNAL: "0.0.0.0:1883"
EMQX_LISTENER__TCP__EXTERNAL__ACCEPTORS: 64
EMQX_LISTENER__TCP__EXTERNAL__MAX_CONNECTIONS: 1024000

Install Chrome OS

Wed, 27 Sep 2023 09:36:00 +0800

老電腦別丟掉！安裝 Google 免費作業系統安裝教學（Chrome OS Flex /CloudReady）

Yongling Foundation AI Forum: How AI Shapes the Future of Humanity

Wed, 27 Sep 2023 09:36:00 +0800

https://www.youtube.com/watch?v=M3z6gI1AEns

Golang Tips

Tue, 26 Sep 2023 22:15:00 +0800

Go Programming Patterns: Slices, Interfaces, Time, and Performance

Go is a high-performance language, but that does not mean we can ignore performance. Here are some tips related to programming and performance.

If you need to convert numbers to strings, strconv.Itoa() is about twice as fast as fmt.Sprintf().
Avoid converting String to []Byte where possible. This conversion hurts performance.
If you use append() on a slice inside a for-loop, expand the slice capacity ahead of time to avoid reallocations and automatic growth by powers of two, which can waste memory.
Use StringBuffer or StringBuild to concatenate strings; they are three to four orders of magnitude faster than + or +=.
Use concurrent goroutines where possible, and use sync.WaitGroup to synchronize concurrent work.
Avoid allocating in hot paths; it makes the GC busy. Use sync.Pool to reuse objects.
Use lock-free operations and avoid mutexes when possible; use sync/Atomic. (See Lock-Free Queue Implementation or Lock-Free Hashmap Implementation.)
Use I/O buffering; I/O is very slow. bufio.NewWrite() and bufio.NewReader() improve performance.
For fixed regex patterns in a for-loop, use regexp.Compile() to precompile; it improves performance by two orders of magnitude.
If you need higher-performance protocols, consider protobuf or msgp instead of JSON, because JSON serialization/deserialization uses reflection.
When using maps, integer keys are faster than string keys because integer comparisons are faster.

datavizproject

Tue, 26 Sep 2023 13:29:00 +0800

https://datavizproject.com

朋友旅行防止絕交檢查表

Tue, 26 Sep 2023 13:29:00 +0800

https://travel-questions.gnehs.net/

Cloudflare Zero Trust

Tue, 26 Sep 2023 09:01:00 +0800

1. Set up the client

Create device enrollment rules

Create device enrollment rules to determine which devices can enroll to Zero Trust organization.

Set device enrollment permissions

In Zero Trust, go to Settings > WARP Client > Device enrollment > Device enrollment permissions > Manage.
Rules > Policies > Add a rule > Include > Selector > Emails ending in > Value > @ruru910.com.

2. Route private network IPs through WARP

In Zero Trust, go to Settings > WARP Client > Device settings > Profile settings > Profile name > Default > Configure.
Configure settings:
1. Enabled: Captive portal detection, Mode switch, Allow device to leave organization, Allow updates.
2. Service mode: Gateway with WARP.
3. Local Domain Fallback > Manage > Domain > nas.ruru910.com.
4. Split Tunnels: Exclude IPs and domains > Manage.
  - Delete the IP range of nas.ruru910.com.

3. Filter network traffic with Gateway

1. Enable the Gateway proxy

In Zero Trust, go to Settings > Network.
1. Gateway Logging: Capture all.
2. Firewall: Proxy(TCP, UDP, ICMP), WARP to WARP, AV inspection.

2. Create Zero Trust policies

Go to Access > Applications > Add an application > Private Network > Application Type > Destination IP.
For Value, enter the IP address for your application (for example, 10.128.0.7).
Modify policy > identify > Selector > User Email > in > @ruru910.com.

Cloudflare Tunnel on Synology

Mon, 25 Sep 2023 22:01:00 +0800

CLOUDFLARE tunnel on SYNOLOGY. (the raw way)

Setup Synology

Create a directory in docker directory, such as cloudflare-tunnel.
Download cloudflared/cloudflared image to registry.
ssh to admin@synology
Change cloudflare-tunnel owner, sudo chown -R 65532:65532 /volume1/docker/cloudflare-tunnel.

Run containers

- `cloudflared tunnel login`

Run container and mount volume docker/cloudflare-tunnel:/home/nonroot/.cloudflared.
Select Use the same network as Docker Host in network tab.
Add command tunnel login in envorinment tab.
Go to container log, and copy login url.
Paste url to browser and authorize the zone.
Export the container setting json to the directory cloudflare-tunnel.

- `cloudflared tunnel create synology-tunnel`

Edit the container setting json in the the directory cloudflare-tunnel, modify cmd. tunnel create synology-tunnel.
Import the container setting json and run a new container.
The container will stop and create tunnel config json in cloudflare-tunnel.
Create config.yml and write ingress rules.
In config.yml, tunnel value is the same as the tunnel config json name, and credentials-file is /home/nonroot/.cloudflared/tunnel config json
Export the second container setting json to the directory cloudflare-tunnel.

- `cloudflared tunnel route dns synology-tunnel synology.ruru910.com`

Edit the second container setting json in the the directory cloudflare-tunnel, modify cmd. tunnel route dns synology-tunnel synology.ruru910.com.
Import the second container setting json and run a new container.
The container will stop and create a dns record mapping domain to the tunnel.

- `cloudflared tunnel run synology-tunnel`

Edit the second container setting json in the the directory cloudflare-tunnel, modify cmd. tunnel run synology-tunnel.
Import the second container setting json and run a new container.
The tunnel now is connectable.

Go Patterns

Sat, 16 Sep 2023 16:08:12 +0800

Faster Multi-Platform Builds: Dockerfile Cross-Compilation Guide

Mon, 04 Sep 2023 10:31:54 +0800

Faster Multi-Platform Builds: Dockerfile Cross-Compilation Guide

method

docker buildx create --use
FROM --platform=linux/amd64 debian / FROM --platform=$BUILDPLATFORM debian
variables

BUILDPLATFORM — matches the current machine. (e.g. linux/amd64)
BUILDOS — os component of BUILDPLATFORM, e.g. linux
BUILDARCH — e.g. amd64, arm64, riscv64
BUILDVARIANT — used to set ARM variant, e.g. v7
TARGETPLATFORM — The value set with --platform flag on build
TARGETOS - OS component from --platform, e.g. linux
TARGETARCH - Architecture from --platform, e.g. arm64
TARGETVARIANT

example

before

FROM golang:1.17-alpine AS build
WORKDIR /src
COPY . .
RUN go build -o /out/myapp .

FROM alpine
COPY --from=build /out/myapp /bin

after

FROM --platform=$BUILDPLATFORM golang:1.17-alpine AS build
WORKDIR /src
ARG TARGETOS TARGETARCH
RUN --mount=target=. \
 --mount=type=cache,target=/root/.cache/go-build \
 --mount=type=cache,target=/go/pkg \
 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o /out/myapp .

FROM alpine
COPY --from=build /out/myapp /bin

How to deal with a 50GB large csv file in r language?

Thu, 20 Jul 2023 15:44:11 +0800

How to deal with a 50GB large csv file in r language?

question

all <- read.csv.ffdf(
 file="<path of large file>",
 sep = ",",
 header=TRUE,
 VERBOSE=TRUE,
 first.rows=10000,
 next.rows=50000,
 )

answer

library(sqldf)

iris2 <- read.csv.sql("iris.csv",
 sql = "select * from file where Species = 'setosa' ")

15 Rules of Channels and Their Implementation

Thu, 22 Jun 2023 21:25:05 +0800

15 Rules of Channels and Their Implementation

Operation rules

Operation	nil	Closed channel	Open buffered channel	Open unbuffered channel
Close	panic	panic	Close succeeds; buffered values can be read. After buffer drains, further reads return the zero value of the channel type.	Close succeeds; further reads return the zero value of the channel type.
Receive	Block	Non-blocking; returns the zero value of the channel type	Non-blocking; reads values normally	Block
Send	Block	panic	Non-blocking; writes values normally	Block

Compile-time rules

Operation	Channel type	Result
Receive	Send-only channel	Compile error
Send	Receive-only channel	Compile error
Close	Receive-only channel	Compile error

Create macOS DMG and Bootable ISO

Mon, 12 Jun 2023 10:04:20 +0800

Create macOS DMG and Bootable ISO

dmg

AppStore ==> search ‘install macos’ ==> get installer ==> du -sh /Applications/Install\ macOS\ Mojave.app/
Disk Utility ==> File ==> New Image ==> Blank Image
1. Size: installer size
2. Format: Mac OS Extended (Journaled)
3. Partition: Single partition - GUID Partition Map
4. Image Format: read/write disk image
5. same as hdiutil create -o ~/Desktop/macOS\ Mojave -size 6500m -layout SPUD -fs HFS+J
sudo /Applications/Install\ macOS\ Mojave.app/Contents/Resources/createinstallmedia --volume /Volumes/macOS\ Mojave

iso

hdiutil convert ~/Desktop/macOS\ Mojave.dmg -format UDTO -o ~/Desktop/macOS\ Mojave.iso
mv ~/Desktop/macOS\ Mojave.iso.cdr ~/Desktop/macOS\ Mojave.iso

Synology: Your NAS RAM Details at a Glance

Mon, 08 May 2023 10:55:23 +0800

Synology: Your NAS RAM Details at a Glance

step1

Control Panel / Task Scheduler / Create / Scheduled Task / User-defined script

step2

General: In the Task field type in RAM Details. Uncheck the “Enabled” option. Select root User.
Schedule: Select Run on the following date then select “Do not repeat”.
Task Settings: Check “Send run details by email”, add your email then copy paste the code below in the Run command area. After that, click OK.

Zsh Config Files and Priority

Mon, 08 May 2023 10:43:10 +0800

Zsh Config Files and Priority

Zsh environment variable config files:

.zprofile
.zlogin
.zshrc
.zshenv
.zlogout

.zshrc is used for overall shell customization

.zprofile and .zlogin are similar. They both set environment variables for login shells, but their load timing differs. .zprofile is based on Bash’s .bash_profile, while .zlogin is the CSH-style .login legacy name.
.zshrc sets environment variables for interactive shells. It loads after .zprofile. Typically used for $PATH, $PROMPT, aliases, functions, etc.
.zshenv is always read, so you can set environment variables like $PATH or $EDITOR there, but it is generally not used.
.zlogout is loaded when a session logs out, which is good for cleanup, such as resetting terminal titles.

Zsh loads ~/.zprofile at login (login shell).

My Productivity Mac Settings and Apps

Thu, 20 Apr 2023 14:52:08 +0800

My Productivity Mac Settings and Apps

Control dark mode per app

https://nightowlapp.co/

brew install --cask nightowl

Control external monitor brightness with the keyboard - MonitorControl

https://github.com/MonitorControl/MonitorControl

brew install --cask monitorcontrol

Raycast

Raycast is a free and more powerful Alfred. You can do most things with the keyboard and quickly launch features with Option + Space. It includes many features that other apps charge for.

Launch apps quickly
Quick dictionary lookup and calculator
Built-in clipboard history search
Window management like Magnet, with shortcuts to resize and move windows
Calendar integration: quickly open Google Meet with the correct account when a meeting starts
Write shell/nodejs/applescript to bind shortcuts for quick actions
Rich extension store

https://www.raycast.com/

Some Software Design Principles

Tue, 18 Apr 2023 13:58:38 +0800

Some Software Design Principles

1. Don’t Repeat Yourself (DRY)

2. Keep It Simple, Stupid (KISS)

Making something complex is easy, but making something complex simple is difficult.

3. Program to an interface, not an implementation

Prefer composition over inheritance
Dependency inversion principle

4. Command-Query Separation (CQS)

Query: when a method returns a value to answer a question
Command: when a method changes the state of an object

5. You Ain’t Gonna Need It (YAGNI)

Only design and implement what is necessary. Avoid overengineering. Implement what you need now; add more when needed later.

On High Availability Systems

Tue, 18 Apr 2023 10:24:00 +0800

On High Availability Systems

Item / Mechanism	Backups	M/S	MM	2PC	Paxos
Consistency	Weak	Eventual	Eventual	Strong	Strong
Transactions	No	Full	Local	Full	Full
Latency	Low	Low	Low	High	High
Throughput	High	High	High	Low	Medium
Data loss	Lots	Some	Some	None	None
Failover	Down	Read only	Read only	Read/write	Read/write

This table essentially covers the foundational solutions for high-availability systems today. M/S and MM are not hard to implement, but have many issues. 2PC suffers from poor performance, while Paxos is too complex and difficult to implement.

Everything About MTU and MSS

Wed, 12 Apr 2023 12:48:12 +0800

Everything About MTU and MSS

Pitfalls When Parsing Binary Data in Go

Fri, 20 Jan 2023 13:48:27 +0800

Pitfalls When Parsing Binary Data in Go

for {
 buf := make([]byte, bufSize)
 n, err := r.Read(buf)
 if err == io.EOF {
 return
 } else if err != nil {
 aerr.Store(err)
 return
 }
 ch <- buf[:n]
}

The code returns immediately when err is io.EOF, which is the most likely problem. The docs say:

Callers should always process the n > 0 bytes returned before considering the error err. Doing so correctly handles I/O errors that happen after reading some bytes and also both of the allowed EOF behaviors.

Rust Easy! Modern Cross-platform Command Line Tools to Supercharge Your Terminal

Fri, 13 Jan 2023 16:51:01 +0800

Rust Easy! Modern Cross-platform Command Line Tools to Supercharge Your Terminal

Windows SSH Setup

Tue, 03 Jan 2023 12:36:00 +0800

The default shell is cmd. The docs say to change the ansible_shell_type variable if needed. This should be set as a host variable in inventory: ansible_shell_type, with value cmd or powershell.
Add the ansible_connection host variable in inventory to indicate SSH connections. (192.168.192.11 ansible_user=Administrator ansible_connection=ssh ansible_shell_type=cmd )
You may need to add remote_tmp in ansible.cfg and set it to C:\TEMP.
In playbooks, use modules prefixed with win_, or use the raw module.

2023 on Ricky

A python script that allows your terminal to snow.

Add SFTP user and share directory

Add SFTP user and share directory

1. 建立共享資料夾(SFTP 使用的資料夾)

2. 建立使用者群組

3. 創建 qa_test_user 使用者並設定 qa_test_user 使用者的群組為 share01-test

4. 創建 qa_user 使用者並設定 qa_user 使用者的群組為 share01-prod

5. 設定權限

6. 設定 /etc/ssh/sshd_config

Use terminal and SSH to remote host

Use terminal and SSH to remote host

1. Modern Terminals

2. Open terminal in macOS

3. SSH to remote host

Fetch GraphQL schema information

Container security fundamentals

Tuning EMQX to Scale to One Million Concurrent Connection on Kubernetes

Linux Kernel Tuning

Erlang VM Tuning

EMQX Broker Tuning

Install Chrome OS

Yongling Foundation AI Forum: How AI Shapes the Future of Humanity

Golang Tips

datavizproject

朋友旅行防止絕交檢查表

Cloudflare Zero Trust

1. Set up the client

Create device enrollment rules

Set device enrollment permissions

2. Route private network IPs through WARP

3. Filter network traffic with Gateway

1. Enable the Gateway proxy

2. Create Zero Trust policies

Cloudflare Tunnel on Synology

Setup Synology

Run containers

- cloudflared tunnel login

- cloudflared tunnel create synology-tunnel

- cloudflared tunnel route dns synology-tunnel synology.ruru910.com

- cloudflared tunnel run synology-tunnel

Go Patterns

Faster Multi-Platform Builds: Dockerfile Cross-Compilation Guide

method

example

How to deal with a 50GB large csv file in r language?

question

answer

15 Rules of Channels and Their Implementation

Operation rules

Compile-time rules

Create macOS DMG and Bootable ISO

dmg

iso

Synology: Your NAS RAM Details at a Glance

step1

step2

Zsh Config Files and Priority

Zsh environment variable config files:

My Productivity Mac Settings and Apps

Control dark mode per app

Control external monitor brightness with the keyboard - MonitorControl

Raycast

Some Software Design Principles

1. Don’t Repeat Yourself (DRY)

2. Keep It Simple, Stupid (KISS)

3. Program to an interface, not an implementation

4. Command-Query Separation (CQS)

5. You Ain’t Gonna Need It (YAGNI)

On High Availability Systems

Everything About MTU and MSS

Pitfalls When Parsing Binary Data in Go

Rust Easy! Modern Cross-platform Command Line Tools to Supercharge Your Terminal

Alacritty

bat

dust

Windows SSH Setup

- `cloudflared tunnel login`

- `cloudflared tunnel create synology-tunnel`

- `cloudflared tunnel route dns synology-tunnel synology.ruru910.com`

- `cloudflared tunnel run synology-tunnel`