List of Linux commands – Useful to regular job
|
SNO
|
Context
|
Command
|
Notes
|
Reference
|
|
1
|
Sum of files in a directory
|
du -ach *
|
Total size of files is shown at the bottom of the output along with individual file size
|
|
|
2
|
Grep a string in dir and sub dir
|
1. grep -R ‘string’ dir/
2. find dir/ -type f -exec grep -H ‘string’ {} +
|
||
|
3
|
Convert date in squid log
|
perl -pe ‘s/\d+/localtime($&)/e’ access.log
|
Squid access.log generally shows in a format of 1178695088.774. This helps to convert it in readable time manner
|
|
|
4
|
Logs between 2 time stamps
|
sed -n ‘/Feb 12 00:00:02/,/Feb 14 23:59:59/P’ access.log
|
||
|
5
|
Count the no’of occurance in a file
|
grep -o ‘503’ access.log |wc -l
|
||
|
6
|
Count the time_wait in server
|
netstat -aon | grep tcp | grep TIME_WAIT | wc -l
|
||
|
7
|
Search for error codes
|
zgrep -P ‘\t503\t’ access_log.20190213* |wc –
|
To search in zip/archived files
|
|
|
8
|
S3 bucket/object size
|
aws s3 ls s3://Bucket-name/ –recursive –human-readable –summarize
|
||
|
9
|
Curl output with timestamp of reqeuest time with new line prints
|
curl -s –insecure -L -X GET -A “ios” –proxy 2.3.4.1:8080 https://wgetip.com -w “\n” | ts ‘[%Y-%m-%d %H:%M:%S]’
|
||
|
10
|
Internet speed test through CLI
|
curl -s https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py | python –
|
For Installation: Download – tool
1. wget -O speedtest-cli https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py
2. chmod +x speedtest-cli
3. ./speedtest-cli
|
|
|
11
|
tcpdump
|
tcpdump -c 8 -tttt -i eth0 -w tcpdump-8080.pcap port 8080
|
||
|
12
|
sbt – package installation
|
2. yum install sbt-0.13.5.rpm
|
SBT – Scala Build Tool – Required for make create on credit stack of iOS proxy
|
|
|
13
|
Compare 2 files and print matched data
|
awk ‘NR==FNR{a[$0];next} ($0 in a)’ file1.out file2.out
|
||
|
14
|
netstat -s
|
|||
|
15
|
netstat -ant | awk ‘{print $6}’ | sort | uniq -c | sort -n
|
|||
|
16
|
History with date
|
1. export HISTTIMEFORMAT=’%F %t ‘
2. history
|
||
|
17
|
sbt version
|
sbt sbtVersion
|
||
|
18
|
Search for a string/text in multiple files
|
grep -rnw ‘/etc/datadog-agent/’ -e ‘squid’
|
||
|
19
|
Extract field based data from a file
|
awk ‘{print $1}’ filename.txt
|
Prints column field in a file
|
|
|
20
|
Print full line of a file based on search field criteria
|
awk ‘$3 == “11903” { print $0 }’ log.txt
|
It prints whole line of a file where ever it has the string value
$3 -> For 3rd field/column
print $0 -> to print whole line
|
|
|
21
|
Prints uniq data from a file
|
awk ‘{print $3} log.txt | sort | uniq
|
print 3rd column/field
|
|
|
22
|
Service monitoring
|
if service is down, it restarts automatically
|
||
|
23
|
Extract 2nd column from a file (Usecase: Log file )
|
awk ‘{print $2}’ error.log > error.out
|
Extract and print in other file
|
|
|
24
|
Print only the chars/string after a delimiter
|
awk -F’.’ ‘{print $1,$2,$3}’ OFS=’.’ error.log
|
Prints only 1st ($1), 2nd and 3rd chars after a delimiter .
|
|
|
25
|
Print number of occurrences in a file while checking the string/char/pattern in one file
|
awk ‘NR==FNR{arr[$0];next} $0 in arr’ file1 file2 | sort | uniq -c
|
It reads line by line in File1 and compare it with File2. If matches prints number of occurrences in File2
|
|
|
26
|
Find files with 0 size and delete them all from a directory
|
|
Linux, DevOps & AWS 