Now, a daily working environment can be build via few lines code on the fly. And effetive, robust, beautiful, consistent on multiple platforms.
There are two ways to use kit:
- clone git repository to your working host
git clone --depth=1 https://github.com/junjiemars/kit.git
- use single kit via bash
$ bash <(curl https://raw.githubusercontent.com/junjiemars/kit/master/<where-the-bash-script>)
Kits for Unix-like platforms.
Locate at ul
directory.
Clone the current shell
environment and the job list of crontab into
a plain text file named crontab.list
.
# make crontab.list file
$ crontab-env.sh
# load crontab.list
crontab crontab.list
A shortcut of exuberant Ctags.
$ ./ctags.sh --help
$ ./ctags.sh --language=c
Download and configure kit via web.
Install java programming environment.
$ HAS_ANT=1 HAS_ABCL=1 HAS_MAVEN=1 ./install-java-kit.sh
Log rotate.
A nginx configurator and shell maker.
Setup bash or zsh, aliases, paths and vars etc., on Windows, Darwin, Linux or Unix-like box, and just one line code you need to getting things done:
$ bash <(curl https://raw.githubusercontent.com/junjiemars/kit/master/ul/sh.sh)
# or zsh
$ $ zsh <(curl https://raw.githubusercontent.com/junjiemars/kit/master/ul/sh.sh)
For Windiows you can use Git Bash instead.
You can boot it from local storage two.
# git clone it from github to <kit-local-dir>
$ git clone --depth=1 https://github.com/junjiemars/kit.git <kit-local-dir>
# boot up from <kit-local-dir>
$ GITHUB_H=file://<kit-local-dir> <kit-local-dir>/ul/sh.sh
Unzip specify code page.
A shortcut of wget to mirror a website.
A tiny-handy Web server based on Netcat (often abbreviated to nc
).
$ ./ws.sh --help
$ ./ws.sh --http-port=8080 --http-version=1.1
$ curl -v http://<host>:8080
Get Windows default code page.
Setup Win32 programming environment.
name | url |
---|---|
aria2c | https://github.com/aria2/aria2/releases |
- Disk slow on Windows 10: Disable write cache in Disk > Hardware > Policies
UEFI(Unified Extensible Firmware Interface) is known as a motherboard firmware interface for computers. With UEFI boot mode, your computer can initialize the hardware components and boot the OS stored on the hard disk.
GPT(GUID Partition Table) is a partitioning schemes that introduced by Intel by the 1990s.
sudo diskutil partitionDisk /dev/disk3 4 MBR \
MS-DOS WIN 20G \
MS-DOS LIN 20G \
MS-DOS MAC 20G \
MS-DOS DTA R
# enter command mode
sudo fdisk -e /dev/disk3
# replace APP_NAME_CASE_SENSITIVE with the app name
sqlite3 \
$(find /private/var/folders \( -name com.apple.dock.launchpad -a -user $USER \) \
2> /dev/null)/db/db \
"DELETE FROM apps WHERE title='APP_NAME_CASE_SENSITIVE';" \
&& killall Dock
- Non-Persistent Routes
# display the entire routing table
netstat -rn
# add non-persistent routes
sudo route -n add 192.168/16 10.0.1.1
# delete non-persistent routes
sudo route -n delete 192.168/16
- Persistent Routes
# display all of the network names
networksetup -listallnetworkservices
# get IP address info for a specific network
networksetup -getinfo "Wi-Fi"
# display routes associated with a specific network
networksetup -getadditionalroutes "Wi-Fi"
# add a static route
networksetup -setadditionalroutes "Wi-Fi" 192.168.0.0 255.255.0.0 10.0.1.1
# delete all static routes
networksetup -setadditionalroutes "Wi-Fi"
/private/etc/hosts
# clean dns cache
sudo killall -HUP mDNSResponder
- Which Port is Listening
$ lsof -iTCP -sTCP:LISTEN -n -P
- Who is Listening on the Port
$ lsof -i :<port> -n -P
/Applications
~/Library
rm -r ~/Library/Saved Application State/${APP}
rm -r ~/Library/Containers/${APP}
rm -r ~/Library/Caches/${APP}
rm -r ~/Library/Application Support/${APP}
rm -r ~/Library/Application Scripts/${APP}
rm -r ~/Library/Preferences/${APP}
- Remove Google software
# local uninstallation:
python ~/Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle/\
Contents/Resources/GoogleSoftwareUpdateAgent.app/Contents/Resources/\
install.py --uninstall
touch ~/Library/Google/GoogleSoftwareUpdate
# global uninstallation:
sudo python /Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle/\
Contents/Resources/GoogleSoftwareUpdateAgent.app/Contents/Resources/\
install.py --uninstall
sudo touch /Library/Google/GoogleSoftwareUpdate
- Remove Oracle JDK
rm -r /Library/Java/JavaVirtualMachines/${JDK_VER}
rm -r /Library/PreferencePanes/JavaControlPanel.prefPane
rm -r /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin
rm -r /Library/Application Support/Oracle
- Install OpenJDK
sudo port install openjdk${JDK_VER}
# JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk%{JDK_VER}/Contents/Home
- Install Wireshark
#+BEING_SRC sh sudo port install wireshark3
wireshark-chmodbpf has the following notes: To fully complete your installation and use Wireshark to capture from devices (like network interfaces) please run:
sudo dseditgroup -q -o edit -a [USER] -t user access_bpf and change [USER] to the user that needs access to the devices. A reboot should not be required for this to take effect.
A startup item has been generated that will start wireshark-chmodbpf with launchd, and will be enabled automatically on activation. Execute the following command to manually disable it:
sudo port unload wireshark-chmodbpf
There are two types of services that launchd manages:
- launch daemon can run without a user logged in.
/System/Library/LaunchDaemons/
for Apple./Library/LaunchDaemons/
for the rest. - launch agent need the user to be logged in.
/System/Library/LaunchAgents/
for Apple./Library/LaunchAgents/
for rest.
launchctl
interfaces with launchd to manage and inspect daemons,
agents and XPC services.
Using Ubuntu as example to explain Linux.
Hardware is the basis.
lshw
lsmod
modprobe
dmesg
Intel CPU
sudo apt install intel-microcode
AMD CPU
sudo apt install amd64-microcode
service --status-all
ls -lh /etc/init.d/
service acpid status
Control the systemd system and service manager.
systemctl show-environment
systemctl list-units
systemctl list-sockets
systemctl list-timers
systemctl list-jobs
systemctl status acpid
/etc/security/limits.conf
, replace user1
with real user name.
* soft nproc 655350 * hard nproc 655350 * soft nofile 655350 * hard nofile 655350 user1 soft nproc 200000 user1 hard nproc 200000 user1 soft nofile 200000 user1 hard nofile 200000
sysctl
# write
sysctl -w vm.swappiness=30
# load
sysctl -p
# check
sysctl -a |*grep swap.*
less /proc/sys/vm/swappiness
IO request follows VFS -> Cache -> Scheduler -> Driver.
iostat
What scheduler?
cat /sys/block/sda/queue/scheduler
Change scheduler.
sudo modprobe kyber-iosched
echo "kyber" | sudo tee /sys/block/sda/queue/scheduler
Persistent scheduler.
- change
/etc/default/grub
GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1 scsi_mod.use_blk_mq=1"
- create
/etc/udev/rules.d/60-scheduler.rules
if it didn’t exist.
ACTION=="add|change", KERNEL=="sda", ATTR{queue/scheduler}="kyber"
- apply
udev
sudo udevadm control --reload
sudo udevadm trigger
vmstat
/proc/sys/vm
(let ((write/s 1200.0) (memory 8000.0))
(let* ((dirty-ratio (* (/ write/s memory) 100))
(background-ratio (ceiling (/ dirty-ratio 2.0))))
(list dirty-ratio background-ratio)))
vm.dirty_background_ratio = 6
vm.dirty_ratio = 12
vm.dirty_writeback_centisecs = 500
vm.dirty_expire_centisecs = 3000
RAM is split into Zones
- Direct Memory Access (DMA): This is the low 16 MB of memory. The zone gets its name because, a long time ago, there were computers that could only do direct memory access into this area of physical memory.
- Direct Memory Access 32: Despite its name, Direct Memory Access 32 (DMA32) is a zone only found in 64-bit Linux. It’s the low 4 GB of memory. Linux running on 32-bit computers can only do DMA to this amount of RAM (unless they are using the physical address extension (PAE) kernel), which is how the zone got its name. Although, on 32-bit computers, it is called HighMem.
- Normal: On 64-bit computers, normal memory is all of the RAM above 4GB (roughly). On 32-bit machines, it is RAM between 16 MB and 896 MB.
- HighMem: This only exists on 32-bit Linux computers. It is all RAM above 896 MB, including RAM above 4 GB on sufficiently large machines.
getconf PAGESIZE
less /proc/zoneinfo
sudo cat /proc/pagetypeinfo
less /proc/buddyinfo
(let ((mem-available 2996044))
(list :write (* (/ mem-available 4 100) 10)
:flood (* (/ mem-available 4 100) 30)))
cpufreqd
# show old
for i in 0 1 2 3; do cat /sys/devices/system/cpu/cpu$i/cpufreq/scaling_governor; done
# trans powersave to performance
# /etc/rc.local
for i in 0 1 2 3; do echo performance > /sys/devices/system/cpu/cpu$i/cpufreq/scaling_governor; done
- Performance Tuning on Linux — Disk I/O
- Performance Tuning on Linux — TCP
- Ubuntu Performance Tuning
- What Is Swappiness on Linux?
- How the Linux kernel divides up your RAM
- Better Linux Disk Caching & Performance with vm.dirty_ratio & vm.dirty_background_ratio
- The Slab Allocator in the Linux kernel
- Arch Linux: Improving performance
Location /var/lib/snapd/snaps/
Just keep the current ul/snap-rm.sh
version.
kazam
: screen recording and capturing program.
Docker daemon run as root user in a group called docker
by default.
$ sudo usermod -a -Gdocker <user>
$ sudo service docker[.io] restart
$ sudo reboot now
Include basic building/networking tools, emacs/vim editors for
c/c++/clang/python/lua development. You can use root or default
sudoer: u/Hell0
to login and play.
*** Build from Dockerfile
$ docker build -t centos-dev https://raw.githubusercontent.com/junjiemars/kit/master/docker/dev/centos.dockerfile
Or you can download centos.dockefile then build from it.
*** Play with centos-dev
Docker Container
- once a time
$ docker run -w /home/u -h centos --privileged -u u -it --rm junjiemars/centos-dev /bin/bash
- as daemon
# gdb or lldb needs privileged permission
$ docker run --name centos-dev -w /home/u -h centos --privileged -d junjiemars/centos-dev
$ docker exec -it -u u centos-dev /bin/bash
- cannot change locale
$ localedef -i en_US -f UTF-8 en_US.UTF-8
** Basic Ubuntu Development Environment
Include basic building/networking tools, emacs/vim editors for
c/c++/llvm/python/lua development. You can use root or default sudoer:
u/Hell0
to login and play.
$ docker build -t ubuntu-dev https://raw.githubusercontent.com/junjiemars/kit/master/docker/dev/ubuntu.dockerfile
Or you can download ubuntu.dockefile then build from it.
- once a time
$ docker run -w /home/u -h ubuntu --privileged -u u -it --rm junjiemars/ubuntu-dev /bin/bash
- as daemon
$ docker run --name ubuntu-dev -w /home/u -h ubuntu --privileged -d junjiemars/ubuntu-dev
$ docker exec -it -u u ubuntu-dev /bin/bash
- Avoid IPv6 if you use a slow tunnel
$ sudo apt-get -o Acquire::ForceIPv4=true [update|install]
- Use mirrors which is based on your geo location
# use mirror automatically
$ sudo cp /etc/apt/sources.list /etc/apt/sources.list.ori
$ sudo sed -i 's#http:\/\/archive.ubuntu.com\/ubuntu\/#mirror:\/\/mirrors.ubuntu.com\/mirrors.txt#' /etc/apt/sources.list
# check mirrors list that based on your geo
$ curl -sL mirrors.ubuntu.com/mirrors.txt
- Aovid posioning mirrors: select another country
Now, the good news is Docker has native stable version for Windows 10
since 7/29/2016. If you need Docker Machine
you can check
DockerMachine on Windows 10.
- Control Panel > Administrative Tools > Hyper-V Manager
- Change
Virtual Machines
location - Change
Virtual Hard Disks
location
- Mintty does not provide full TTY support;
- Use
cmd
orPowerShell
;
- Hyper-V Manager: keep only one
MobiLinuxVM
and delete all the others
$ docker build -t java-dev https://raw.githubusercontent.com/junjiemars/kit/master/docker/dev/java.dockerfile
- one time
$ docker run -w /home/u -h centos -u u -it --rm java-dev /bin/bash
- as daemon
$ docker run --name java-dev -w /home/u -h centos --privileged -d java-dev
$ docker exec -it -u u java-dev /bin/bash
Run into java-dev container and then run install-java-kits.sh
$ HAS_ALL=YES install-java-kits.sh
You can pull it from Docker Hub, or build it from oracle_xe.dockerfile. Beside, Oracle XE 11g2 could not be downloaded via curl, there needs some hack way to do it.
$ docker pull junjiemars/xe11g2:latest
$ docker run --name xe11g2 -p 1521:1521 -p 8080:9000 -d junjiemars/xe11g2:latest
- Install Docker Toolbox
- Run Docker Quickstart Terminal
- Play docker, it’s same on Linux box
- Kitematic is useless, don’t install it
- Need VirtualBox and NIS6+
Font
: On Windows, the Console’s font is ugly if the code page is 936 for Chinese locale. Change the Windows locale to English and change the font to Consolas or others thats good for English lauguage. Restart Windows then switch the locale back to your locale, then restart it again.Mintty
: Mintty is not based on Windows’ Console, it’s better than git-bash. To use Mintty via change Docker Quickstart Terminal’s the target inshortcut
to
"C:\Program Files\Git\usr\bin\mintty.exe" -i "c:\Program Files\Docker Toolbox\docker-quickstart-terminal.ico" /usr/bin/bash --login -i "c:\Program Files\Docker Toolbox\start.sh"
MACHINE_STORAGE_PATH
: Environment variable points to docker’s image location.
- Configure
Shared folders
on VirtualBox:
<vbox-folder-label-name> -> <windows-local-dir>
- Mount the dir on Docker VM:
$ docker-machine ssh [machine-name]
$ mkdir -p /home/docker/<dir-name>
$ sudo mount -t vboxsf -o uid=1000,gid=50 <vbox-folder-label-name> /home/docker/<dir-name>
- Run Docker Host with
Volume
:
$ docker run -d -v <vbox-folder-label-name>:<docker-host-mount-dir> <image>
If you got cannot enable tty mode on non tty input
, so
$ docker-machine ssh <default>
- machine -> host:
$ docker-machine scp <machine>:<machine-path> <host-path>
- host -> machine:
$ docker-machine scp <host-path> <machine>:<machine-path>
- container -> host
# copy from container to machine
$ docker cp <container-path> <machine-path>
# copy from machine to host
$ docker-machine scp <machine>:<machine-path> <host-path>
- host -> container vice versa.
The default docker0
virtual bridge interface let communications:
- container -> container
- container -> host
- host -> container
Read from socket failed: Connection reset by peer
$ sudo ssh-keygen -t rsa -f /etc/ssh/ssh_hosts_rsa_key
$ sudo ssh-keygen -t dsa -f /etc/ssh/ssh_hosts_dsa_key
- Container’s IP address
# on default bridge network
$ docker inspect --format "{{.NetworkSettings.IPAddress}}" <container-id|container-name>
# on specified network
docker inspect --format "{{.NetworkSettings.Networks.<your-network>.IPAddress}}" <container-id|container-name>
- Link to Another Containers (/etc/hosts)
$ docker run --name n2 --link=n0 --link=n1 -d <docker-image>
"registry-mirrors": [
"https://registry.docker-cn.com",
"https://docker.mirrors.ustc.edu.cn"
]
# create mount the volume on /opt/vol
$ docker run --name n0 -w /home/u -h n0 -v /opt/vol -d <docker-iamge>
# mount a host volume on /opt/vol
$ docker run --name n0 -w /home/u -h n0 -v <host-path>:/opt/vol -d <docker-image>
# mount a host file
$ docker run --name n0 -w /home/u -h n0 -v ~/.bash_history:/home/u/.bash_history -d <docker-image>
cd kit/podman
make -C compose -f <xxx.Makefile> up
# install podman
sudo apt install podman podman-gvproxy podman-plugins
# install podman-compose
sudo pip3 install podman-compose
# install CNI
Login with user ubuntu
and default password ubuntu
, then prompt to
change the default password.
sudo diskutil list
sudo diskutil unmountdisk /dev/xxx
xzcat ubuntu.img.xz | sudo dd of=/dev/xxx bs=4M
sudo vi /etc/netplan/50-cloud-init.yaml
network:
version: 2
ethernets:
eth0:
dhcp4: no
addresses: [192.168.1.15/24,]
gateway4: 192.168.1.1
nameservers:
addresses: [208.67.222.222,8.8.8.8]
match:
macaddress: xx:xx:xx:xx:xx:xx
set-name: eth0
sudo netplan apply
make -C docker/compose/ -f oraclexe.Makefile start
You can using raw sqlplus
command program or an awsome tool
sqlplus.sh
, or Emacs.
List tables
select name from sqlite_master where type='table';
Table structure #+BEING_SRC sql pragma table_info(<table>);
make -C docker/compose/ -f mysql.Makefile start
mysql> select @@sql_mode;
mysql> SET GLOBAL sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''));
mysql> SET GLOBAL sql_mode = sys.list_drop(@@GLOBAL.sql_mode, 'ONLY_FULL_GROUP_BY');
mysql> SET @@sql_mode = @@GLOBAL.sql_mode;
SELECT [Name],
GROUP_CONCAT(DISTINCT [Name] SEPARATOR ',')
FROM tbl_cars
git config --global http.proxy 'socks5://127.0.0.1:<port>'
# or per-user: export ALL_PROXY='socks5://127.0.0.1:<port>'
# git something...
git config --global --unset http.proxy
fix the .git/config
file
[remote "origin"]
url = https://github.com/junjiemars/kit.git
fetch= +refs/heads/*:refs/remotes/origin/*
On Windows, svn
doesn’t support posix repository URL.
# create repository
svnadmin create </opt/lab/vcs/svn/repo>
# make hooks/pre-revprop-change file
chmod u+x hooks/pre-revprop-change
# checkout repository
svn checkout <file://E:/lab/vcs/svn/repo>
# update local
svn update
# commit
svn add .
svn commit -m"message"
# log
svn log --limit 8
hooks/pre-revprops-change
#!/bin/sh
exit 0;
Clone the SVN repository
git svn clone <file:///a/b/c>
Create a new branch to modify or work on a new feature
git checkout -b <new-branch>
Work on your newly created branch in the local repository as usual. Once complete, you can switch to the master branch and merge your changes to master.
git checkout master
git merge <new-branch>
Update your SVN repository
git svn rebase
Commit back to the SVN repository
git svn dcommit -m"commit back"
# VER=openjdk-11, default version
sudo apt update
sudo install ${VER}-jdk ${VER}-dbg ${VER}-11-doc
sudo update-alternatives --install /usr/bin/java, java `echo $JAVA_HOME`/bin/java
sudo update-alternatives --install /usr/bin/javac, javac `echo $JAVA_HOME`/bin/javac
sudo update-alternatives --install /usr/bin/jhsdb, jhsdb `echo $JAVA_HOME`/bin/jhsdb
jps -vl
jhsdb -h
jhsdb jstack -h
echo $PASSWORD | sudo -S jhsdb jmap --pid $PID
Allow jdb
to attach process on Ubuntu.
* --module-path /Users/<user>/Downloads/javafx-sdk-11/lib --add-modules=javafx.controls,javafx.fxml
NODE_OPTIONS
environment variable
export NODE_OPTIONS="--max-old-space-size=1024"
# generate a private key and a CSR(Certificate Signing Request)
openssl req -newkey rsa:2048 -nodes -keyout x.key -out x.csr
openssl req \
-newky rsa:2048 -nodes -keyout x.key \
-x509 -days 365 -out z.crt
keytool -v -list -keystore <key-store>
Find the configuration file: idea64.vmoptions
at
~/.config/JetBrains
.
-Xms128m
-Xmx1024m
-XX:+UseParallelGC
-XX:+UseAdaptiveSizePolicy
-XX:-UseConcMarkSweepGC
-Xms
: specify the initial memory allocation pool.-Xmx
: specify the maximum memory allocation pool.+UseParallelGC
: faster responsibilies but may be unstable-XX:+UseParallelGC
: faster responsibilies but may be unstable-XX:+UseAdaptiveSizePolicy
: faster responsibilies but may be unstable-XX:-UseConcMarkSweepGC
: faster responsibilies but may be unstable
Default gradle home at ~/.gradle
.
Open File=>=Settings=>=
idea-jar-kit.sh import/export settings.
# export to kit
./idea-jar-kit.sh -x=.
# import from kit
./idea-jar-kit.sh -a=.
# list
./idea-jar-kit.sh -l=