Reminder

Table of Contents

1. Bash shell
2. Vim
3. Kafka
4. Python
5. GIT
6. API
7. CheatSheet
8. ffmeg

Bash shell

Bash string manipulation cheatsheet

Assignment
Assign value to variable if variable is not already set, value is returned. Combine with a : no-op to discard/ignore return value.	${variable="value"} : ${variable="value"}
Removal
Delete shortest match of needle from front of haystack	${haystack#needle}
Delete longest match of needle from front of haystack	${haystack##needle}
Delete shortest match of needle from back of haystack	${haystack%needle}
Delete longest match of needle from back of haystack	${haystack%%needle}
Replacement
Replace first match of needle with replacement from haystack	${haystack/needle/replacement}
Replace all matches of needle with replacement from haystack	${haystack//needle/replacement}
If needle matches front of haystack replace with replacement	${haystack/#needle/replacement}
If needle matches back of haystack replace with replacement	${haystack/%needle/replacement}
Substitution
If variable not set, return value, else variable	${variable-value}
If variable not set or empty, return value, else variable	${variable:-value}
If variable set, return value, else null string	${variable+value}
If variable set and not empty, return value, else null string	${variable:+value}
Extraction
Extract length characters from variable starting at position	${variable:position:length}
Return string length of variable	${#variable}
Escaping
Single quotes inside a single quoted string	echo 'Don'\''t break my escape!'
Indirection
Return value of variable name held in indirect, else value	indirect="apple" apple="fruit" ${!indirect-value}

Reference

• https://tldp.org/LDP/abs/html/string-manipulation.html
• https://tldp.org/LDP/abs/html/parameter-substitution.html
• https://tldp.org/LDP/abs/html/ivr.html
• Special characters:
  • *: https://www.tldp.org/LDP/abs/html/special-chars.html#ASTERISKREF
  • ?: https://www.tldp.org/LDP/abs/html/special-chars.html#WILDCARDQU
• https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02

Reference

• The programmer Stone
• https://catonmat.net/the-definitive-guide-to-bash-command-line-history
• http://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html
• https://linuxhint.com/30_bash_script_examples/
• https://dev.to/awwsmm/101-bash-commands-and-tips-for-beginners-to-experts-30je#click-click
• https://catonmat.net/bash-one-liners-explained-part-two

Password generation

• openssl rand -base64 32
• tr -cd '[:alnum:]' < /dev/urandom | fold -w30 | head -n1
• < /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c${1:-32};echo;

bash history manipulation

• !!:gs/search/replace/
• ^x^y

Retrieve my IP

• ip=$(hostname --ip-address)
• ip=$(ip route get 1 | awk '{print $NF;exit}')
• curl https://ipinfo.io/ip
• wget -qO- https://ipecho.net/plain ; echo
• dig +short myip.opendns.com @resolver1.opendns.com
• LANG=c ifconfig | grep -B1 "inet addr" |awk '{ if ( $1 == "inet" ) { print $2 } else if ( $2 == "Link" ) { printf "%s:" ,$1 } }' |awk -F: '{ print $1 ": " $3 }'
• http://checkip.amazonaws.com
• ifconfig.me
• stunclient stun.services.mozilla.com
• ip addr show

AWK tips

• Extract Value beetween tags : echo "asdf <ip>test</ip> dsaf" | awk -v RS='</?ip>' '!(NR%2)'
  • Result : test

Check port open

• (timeout 2 bash -c '</dev/tcp/127.0.0.1/17500 && echo PORT OPEN || echo PORT CLOSED') 2>/dev/null
• nc -zv kafka02 6667 -z = sets nc to simply scan for listening daemons, without actually sending any data to them -v = enables verbose mode

Ping without ping (Test port with no tools)

• timeout 10 true >/dev/tcp/8.8.8.8/53
• timeout 10 true >/dev/tcp/142.250.201.163/80
• timeout 10 true >/dev/tcp/142.250.201.163/809

Bash scripting

default argument

• somecommand ${1:-foo}
• ternaire operator [[ $vartest = "data" ]] && var="iftrue" || var="iffalse"

check

#!/bin/bash
set -eo pipefail

Confirm manipulation

read -n 1 -p "Apply this ? (y/n) " answer case ${answer:0:1} in y|Y ) do_works ;; * ) echo "Cancel" ;; esac

Color manipulation

red=`tput setaf 1`
green=`tput setaf 2`
yellow=`tput setaf 3`
bold=`tput bold`
underline=`tput smul`
reset=`tput sgr0`
echo "${underline}${bold}${red}Start Script ...${reset}"

sed

Replace dos to unix

sed -i -e 's/\r$//' myscript.sh

TLS / Certificats

CSR : Certificat Server Request

• openssl req -new -newkey rsa:2048 -nodes -keyout mydomain.key -out mydomain.csr
• openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr -subj "/C=Country/ST=State or Province/L=Locality or City/O=Company/OU=Organizational unit/CN=Common Name"

Check certificat / private key / csr

• openssl rsa -noout -modulus -in mykey.key | openssl md5 > key.mod

• openssl req -noout -modulus -in mycsr.csr | openssl md5 > csr.mod

• openssl x509 -noout -modulus -in mycert.crt | openssl md5 > cert.mod

• diff3 key.mod cert.mod csr.mod Result must be the same.

• https://www.sysnove.fr/blog/2016/03/utilisation-pratique-letsencrypt-acme-tiny.html

Vim

• :w !sudo tee %

 :w = Write a file.
 !sudo = Call shell sudo command.
 tee = The output of the vi/vim write command is redirected using tee.
 % = Triggers the use of the current filename.
 Simply put, the ‘tee’ command is run as sudo and follows the vi/vim command on the current filename given.

• :s/foo/bar/G
• :r myfile insert the content of myfile in the open buffer

Kafka

References

• https://riptutorial.com/fr/apache-kafka/example/27964/kafka-topics

Command

• Topics list:

kafka-topics  --zookeeper localhost:2181 --list

• Topics list:

kafka-topics  --bootstrap-server localhost:9092 --list

• Create topics:

kafka-topics  --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

• Describe topics:

kafka-topics  --zookeeper localhost:2181 --describe --topic test

• Change topics:

change configuration
kafka-topics  --zookeeper localhost:2181 --alter --topic test --config max.message.bytes=128000
# add a partition
kafka-topics  --zookeeper localhost:2181 --alter --topic test --partitions 2

Python

python oneliner

• urllib.request.urlopen('http://www.python.org/')
• python2 : python -m SimpleHTTPServer 8000
• python3 : python -m http.server 8000 [--bind 127.0.0.1 --directory /tmp/ --cgi 8000]
• debug with breakpoint()

python pip

• pip install --download mypackage
• pip install --no-index --find-links pwd mypackage
• python pip-10.0.1-py2.py3-none-any.whl/pip install --no-index pip-10.0.1-py2.py3-none-any.whl

GIT

• git config --list
• git config --global user.name "John Doe"
• git config --global user.email [jd@polop]

API

https://www.ipify.org/

Exemple :

curl 'https://api.ipify.org?format=json' 
{"ip":"164.128.235.36"}

CheatSheet

• http://overapi.com/javascript
• https://www.debuggex.com/cheatsheet/regex/
• https://marozed.ma/vue-cheatsheet/
• https://vuejs-tips.github.io/vuex-cheatsheet/
• https://yoksel.github.io/flex-cheatsheet/
• https://devhints.io/
• https://raw.githubusercontent.com/sogko/graphql-shorthand-notation-cheat-sheet/master/graphql-shorthand-notation-cheat-sheet.png
• https://kubernetes.io/fr/docs/reference/kubectl/cheatsheet/

Coding Interview

• https://realpython.com/python-coding-interview-tips/
• https://github.com/lydiahallie/javascript-questions

FFMEG

• Convertion en mp3 d'une autre source : ffmpeg -i audio.aac -acodec libmp3lame audio.mp3