the-book-of-secret-knowledge/README.md
trimstray 177be60ced added urlscan.io
- signed-off-by: trimstray <trimstray@gmail.com>
2018-07-16 21:09:53 +02:00

1729 lines
47 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<p align="center">
<img src="https://github.com/trimstray/awesome-ninja-admins/blob/master/doc/img/awesome_ninja_admins.png"
alt="Master">
</p>
<br>
<h4 align="center">A collection of awesome lists, manuals, blogs, hacks, one-liners and tools for <b>Awesome Ninja Admins</b>.</h4>
<br>
<p align="center">
<a href="https://github.com/trimstray/awesome-ninja-admins/tree/master">
<img src="https://img.shields.io/badge/Branch-master-green.svg?longCache=true"
alt="Branch">
</a>
<a href="https://awesome.re">
<img src="https://awesome.re/badge.svg"
alt="Awesome">
<a href="http://www.gnu.org/licenses/">
<img src="https://img.shields.io/badge/License-GNU-blue.svg?longCache=true"
alt="License">
</a>
</p>
<div align="center">
<sub>Created by
<a href="https://twitter.com/trimstray">trimstray</a> and
<a href="https://github.com/trimstray/awesome-ninja-admins/graphs/contributors">
contributors
</a>
</div>
<br>
***
## Who is Ninja Admins?
- race of pure evil who rule the network through a monarchistic feudelic system
- they never opened the door for strangers (or anyone at all)
- they know very nasty piece of code like a **[fork bombs](https://en.wikipedia.org/wiki/Fork_bomb)**
- they can make dd is not a **[destroyer of disks](http://www.noah.org/wiki/Dd_-_Destroyer_of_Disks)**
- they know that `#!/usr/bin/env bash` superior to `#!/bin/bash`
- they know that `su -` logs in completely as root
- they miss and cry for **[Slackware](http://www.slackware.com/)** on production
- they love the old admin nix-world
## :ballot_box_with_check: Todo
- [ ] Add useful shell functions
- [ ] Add one-liners for collection tools (eg. CLI Tools)
- [ ] Add Ninja Admins T-Shirt stickers
- [ ] Generate Awesome Ninja Admins book (eg. pdf format)
## Ninja Admins Collection
#### CLI Tools
##### :black_small_square: Shells
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://ohmyz.sh/"><b>Oh My ZSH!</b></a> - the best framework for managing your Zsh configuration.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/Bash-it/bash-it"><b>bash-it</b></a> - framework for using, developing and maintaining shell scripts and custom commands for your daily work.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/oh-my-fish/oh-my-fish"><b>Oh My Fish</b></a> - the Fishshell framework.<br>
</p>
##### :black_small_square: Managers
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://midnight-commander.org/"><b>Midnight Commander</b></a> - visual file manager, licensed under GNU General Public License.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.gnu.org/software/screen/"><b>screen</b></a> - full-screen window manager that multiplexes a physical terminal.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/tmux/tmux/wiki"><b>tmux</b></a> - terminal multiplexer, lets you switch easily between several programs in one terminal.<br>
</p>
##### :black_small_square: Network
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://curl.haxx.se/"><b>Curl</b></a> - command line tool and library
for transferring data with URLs.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/jkbrzt/httpie"><b>HTTPie</b></a> - a user-friendly HTTP client.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://gnutls.org/manual/html_node/gnutls_002dcli-Invocation.html"><b>gnutls-cli</b></a> - client program to set up a TLS connection to some other computer.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://nmap.org/"><b>nmap</b></a> - free and open source (license) utility for network discovery and security auditing.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://www.hping.org/"><b>hping</b></a> - command-line oriented TCP/IP packet assembler/analyzer.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/robertdavidgraham/masscan"><b>masscan</b></a> - the fastest Internet port scanner, spews SYN packets asynchronously.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://netcat.sourceforge.net/"><b>netcat</b></a> - networking utility which reads and writes data across network connections, using the TCP/IP protocol.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.tcpdump.org/"><b>tcpdump</b></a> - powerful command-line packet analyzer.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.wireshark.org/docs/man-pages/tshark.html"><b>tshark</b></a> - dump and analyze network traffic (wireshark cli).<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/mschwager/fierce"><b>fierce</b></a> - a DNS reconnaissance tool for locating non-contiguous IP space.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/aboul3la/Sublist3r"><b>sublist3r</b></a> - fast subdomains enumeration tool for penetration testers.<br>
</p>
##### :black_small_square: SSL
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/nabla-c0d3/sslyze"><b>sslyze
</b></a> - fast and powerful SSL/TLS server scanning library.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/rbsec/sslscan"><b>sslscan</b></a> - tests SSL/TLS enabled services to discover supported cipher suites.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/drwetter/testssl.sh"><b>testssl.sh</b></a> - testing TLS/SSL encryption anywhere on any port.<br>
</p>
##### :black_small_square: Auditing Tools
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://cisofy.com/lynis/"><b>lynis</b></a> - battle-tested security tool for systems running Linux, macOS, or Unix-based operating system.<br>
</p>
##### :black_small_square: System Diagnostics/Debuggers
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/strace/strace"><b>strace</b></a> - diagnostic, debugging and instructional userspace utility for Linux.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/draios/sysdig"><b>sysdig</b></a> - system exploration and troubleshooting tool with first class support for containers.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://en.wikipedia.org/wiki/Lsof"><b>lsof</b></a> - displays in its output information about files that are opened by processes.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://www.brendangregg.com/flamegraphs.html"><b>FlameGraph</b></a> - stack trace visualizer.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/zevv/lsofgraph"><b>lsofgraph</b></a> - small utility to convert Unix lsof output to a graph showing FIFO and UNIX interprocess communication.<br>
</p>
##### :black_small_square: Log Analyzers
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://goaccess.io/"><b>GoAccess</b></a> - real-time web log analyzer and interactive viewer that runs in a terminal.<br>
</p>
##### :black_small_square: Databases
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/xo/usql"><b>usql</b></a> - universal command-line interface for SQL databases.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/dbcli/pgcli"><b>pgcli</b></a> - postgres CLI with autocompletion and syntax highlighting.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/dbcli/mycli"><b>mycli</b></a> - terminal client for MySQL with autocompletion and syntax highlighting.<br>
</p>
#### Web Tools
##### :black_small_square: SSL
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.ssllabs.com/ssltest/"><b>SSL Server Test</b></a> - free online service performs a deep analysis of the configuration of any SSL web server.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://dev.ssllabs.com/ssltest/"><b>SSL Server Test (DEV)</b></a> - free online service performs a deep analysis of the configuration of any SSL web server.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.htbridge.com/ssl/"><b>ImmuniWeb® SSLScan</b></a> - test SSL/TLS (PCI DSS, HIPAA and NIST).<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://urlscan.io/"><b>urlscan.io</b></a> - service to scan and analyse websites.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://report-uri.com/home/tools"><b>Report URI</b></a> - monitoring security policies like CSP and HPKP.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://csp-evaluator.withgoogle.com/"><b>CSP Evaluator</b></a> - allows developers and security experts to check if a Content Security Policy.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://badssl.com/"><b>badssl.com</b></a> - memorable site for testing clients against bad SSL configs.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://ccadb.org/resources"><b>Common CA Database</b></a> - repository of information about CAs, and their root and intermediate certificates.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://certstream.calidog.io/"><b>CERTSTREAM</b></a> - real-time certificate transparency log update stream.<br>
</p>
##### :black_small_square: HTTP Headers
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://securityheaders.com/"><b>Security Headers</b></a> - analyse the HTTP response headers (with rating system to the results).<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://observatory.mozilla.org/"><b>Observatory by Mozilla</b></a> - set of tools to analyze your website.<br>
</p>
##### :black_small_square: DNS
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://viewdns.info/"><b>ViewDNS</b></a> - one source for free DNS related tools and information.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://dnsspy.io/"><b>DNS Spy</b></a> - monitor, validate and verify your DNS configurations.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://dnslytics.com/"><b>DNSlytics</b></a> - online investigation tool.<br>
</p>
##### :black_small_square: Mail
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://mxtoolbox.com/SuperTool.aspx"><b>MX Toolbox</b></a> - all of your MX record, DNS, blacklist and SMTP diagnostics in one integrated tool.<br>
</p>
##### :black_small_square: Mass scanners (search engines)
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://censys.io/"><b>Censys</b></a> - platform that helps information security practitioners discover, monitor, and analyze devices.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.shodan.io/"><b>Shodan</b></a> - the world's first search engine for Internet-connected devices.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://viz.greynoise.io/table"><b>GreyNoise</b></a> - mass scanner (such as Shodan and Censys).<br>
</p>
##### :black_small_square: Net-tools
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://toolbar.netcraft.com/site_report"><b>Netcraft</b></a> - detailed report about the site, helping you to make informed choices about their integrity.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://securitytrails.com/"><b>Security Trails</b></a> - APIs for Security Companies, Researchers and Teams.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://tools.keycdn.com/curl"><b>Online Curl</b></a> - curl test, analyze HTTP Response Headers.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://ping.eu/"><b>Ping.eu</b></a> - online Ping, Traceroute, DNS lookup, WHOIS and others.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://network-tools.com/"><b>Network-Tools</b></a> - network tools for webmasters, IT technicians & geeks.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.url-encode-decode.com/"><b>URL Encode/Decode</b></a> - tool from above to either encode or decode a string of text.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://regexr.com/"><b>RegExr</b></a> - online tool to learn, build, & test Regular Expressions (RegEx / RegExp).<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.hardenize.com/"><b>Hardenize</b></a> - deploy the security standards.<br>
</p>
##### :black_small_square: Code parsers/playgrounds
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.shellcheck.net/"><b>ShellCheck</b></a> - finds bugs in your shell scripts.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://jsbin.com/?html,output"><b>jsbin</b></a> - live pastebin for HTML, CSS & JavaScript and more.<br>
</p>
##### :black_small_square: Performance
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://gtmetrix.com/"><b>GTmetrix</b></a> - analyze your sites speed and make it faster.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://performance.sucuri.net/"><b>Sucuri loadtimetester</b></a> - test here the
performance of any of your sites from across the globe.<br>
</p>
##### :black_small_square: Passwords
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.random.org/passwords/"><b>Random.org</b></a> - generate random passwords.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://gotcha.pw/"><b>Gotcha?</b></a> - list of 1.4 billion accounts circulates around the Internet.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://haveibeenpwned.com/"><b>have i been pwned?</b></a> - check if you have an account that has been compromised in a data breach.<br>
</p>
#### Manuals/Howtos/Tutorials
##### :black_small_square: Bash
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/dylanaraps/pure-bash-bible"><b>pure-bash-bible</b></a> - a collection of pure bash alternatives to external processes.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://wiki.bash-hackers.org/start"><b>The Bash Hackers Wiki</b></a> - hold documentation of any kind about GNU Bash.<br>
</p>
##### :black_small_square: Unix tutorials
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.cyberciti.biz/"><b>nixCraft</b></a> - linux and unix tutorials for new and seasoned sysadmin.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.tecmint.com/"><b>TecMint</b></a> - the ideal Linux blog for Sysadmins & Geeks.<br>
</p>
##### :black_small_square: Hacking
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://www.hackingarticles.in/"><b>Hacking Articles</b></a> - LRaj Chandel's Security & Hacking Blog.<br>
</p>
#### Blogs
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://www.brendangregg.com/"><b>Brendan Gregg's Blog</b></a> - Brendan Gregg is an industry expert in computing performance and cloud computing.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://gynvael.coldwind.pl/"><b>Gynvael "GynDream" Coldwind</b></a> - Gynvael is a IT security engineer at Google.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://lcamtuf.coredump.cx/"><b>Michał "lcamtuf" Zalewski</b></a> - "white hat" hacker, computer security expert.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://ma.ttias.be/"><b>Mattias Geniar</b></a> - developer, Sysadmin, Blogger, Podcaster and Public Speaker.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://nickcraver.com/"><b>Nick Craver</b></a> - Software Developer and Systems Administrator for Stack Exchange.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://robert.penz.name/"><b>Robert Penz</b></a> - IT security Expert.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://scotthelme.co.uk/"><b>Scott Helme</b></a> - Security Researcher, international speaker and founder of securityheaders.com and report-uri.com.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://security.szurek.pl/"><b>Kacper Szurek</b></a> - Detection Engineer at ESET.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.troyhunt.com/"><b>Troy Hunt</b></a> - Microsoft Regional Director and Microsoft Most Valuable Professional for Developer Security.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://linux-audit.com/"><b>Linux Audit</b></a> - the Linux security blog about Auditing, Hardening and Compliance by Michael Boelen.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://linuxsecurity.expert/"><b>
Linux Security Expert</b></a> - trainings, howtos, checklists, security tools and more.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://www.grymoire.com/"><b>The Grymoire</b></a> - collection of useful incantations for wizards, be you computer wizards, magicians, or whatever.<br>
</p>
#### Systems/Services
##### :black_small_square: Systems
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://www.slackware.com/"><b>Slackware</b></a> - the most "Unix-like" Linux distribution.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.openbsd.org/"><b>OpenBSD</b></a> - multi-platform 4.4BSD-based UNIX-like operating system.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://hardenedbsd.org/"><b>HardenedBSD</b></a> - HardenedBSD aims to implement innovative exploit mitigation and security solutions.<br>
</p>
##### :black_small_square: HTTP(s) Services
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://varnish-cache.org/"><b>Varnish HTTP Cache</b></a> - HTTP accelerator designed for content-heavy dynamic web sites.<br>
</p>
##### :black_small_square: Security/hardening
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://emeraldonion.org/"><b>Emerald Onion</b></a> - seattle-based encrypted-transit internet service provider.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.unbound.net/"><b>Unbound</b></a> - validating, recursive, and caching DNS resolver (with TLS).<br>
</p>
#### Lists
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/kahun/awesome-sysadmin"><b>Awesome Sysadmin</b></a> - amazingly awesome open source sysadmin resources.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/alebcay/awesome-shell"><b>Awesome Shell</b></a> - awesome command-line frameworks, toolkits, guides and gizmos.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/Hack-with-Github/Awesome-Hacking"><b>Awesome-Hacking</b></a> - awesome lists for hackers, pentesters and security researchers.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/learnbyexample/Command-line-text-processing"><b>Command-line-text-processing</b></a> - from finding text to search and replace, from sorting to beautifying text and more.<br>
</p>
#### Hacking/Penetration testing
##### :black_small_square: Bounty programs
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.openbugbounty.org/"><b>Openbugbounty</b></a> - allows any security researcher reporting a vulnerability on any website.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.hackerone.com/"><b>hackerone</b></a> - global hacker community to surface the most relevant security issues.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.bugcrowd.com/"><b>bugcrowd</b></a> - crowdsourced cybersecurity for the enterprise.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://crowdshield.com/"><b>Crowdshield</b></a> - crowdsourced Security & Bug Bounty Management.<br>
</p>
##### :black_small_square: Web Training Apps
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://metasploit.help.rapid7.com/docs/metasploitable-2"><b>Metasploitable 2</b></a> - vulnerable web application amongst security researchers.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="http://www.dvwa.co.uk/"><b>DVWA</b></a> - PHP/MySQL web application that is damn vulnerable.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://sourceforge.net/projects/mutillidae/"><b>OWASP Mutillidae II</b></a> - free, open source, deliberately vulnerable web-application.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.owasp.org/index.php/OWASP_Juice_Shop_Project"><b>OWASP Juice Shop Project</b></a> - the most bug-free vulnerable application in existence.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.owasp.org/index.php/Category:OWASP_WebGoat_Project"><b>OWASP WebGoat Project</b></a> - insecure web application maintained by OWASP designed to teach web app security.<br>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/opendns/Security_Ninjas_AppSec_Training"><b>Security Ninjas</b></a> - open source application security training program.<br>
</p>
##### :black_small_square: Labs
<p>
&nbsp;&nbsp;:small_orange_diamond: <a href="https://www.hackthebox.eu/"><b>Hack The Box</b></a> - online platform allowing you to test your penetration testing skills.<br>
</p>
#### One-liners
##### Table of Contents
- **[System](#system)**
* [terminal](#tool-terminal)
* [mount](#tool-mount)
* [fuser](#tool-fuser)
* [ps](#tool-ps)
* [top](#tool-top)
* [kill](#tool-kill)
* [find](#tool-find)
* [diff](#tool-diff)
* [tail](#tool-tail)
* [cpulimit](#tool-cpulimit)
* [pwdx](#tool-pwdx)
* [tr](#tool-tr)
* [chmod](#tool-chmod)
* [who](#tool-who)
* [screen](#tool-screen)
* [du](#tool-du)
* [inotifywait](#tool-inotifywait)
* [openssl](#tool-openssl)
* [gnutls-cli](#tool-gnutls-cli)
* [secure-delete](#tool-secure-delete)
* [dd](#tool-dd)
- **[HTTP/HTTPS](#http-https)**
* [curl](#tool-curl)
* [httpie](#tool-httpie)
- **[Network](#network)**
* [ssh](#tool-ssh)
* [linux-dev](#tool-linux-dev)
* [tcpdump](#tool-tcpdump)
* [tcpick](#tool-tcpick)
* [ngrep](#tool-ngrep)
* [hping3](#tool-hping3)
* [netcat](#tool-netcat)
* [socat](#tool-socat)
* [lsof](#tool-lsof)
* [netstat](#tool-netstat)
* [rsync](#tool-rsync)
* [host](#tool-host)
* [dig](#tool-dig)
* [network-other](#tool-network-other)
- **[Programming](#programming)**
* [awk](#tool-awk)
* [sed](#tool-sed)
* [grep](#tool-grep)
<a name="system"><b>System</b></a>
##### Tool: [terminal](https://en.wikipedia.org/wiki/Linux_console)
###### Close shell keeping all subprocess running
```bash
disown -a && exit
```
###### Exit without saving shell history
```bash
kill -9 $$
unset HISTFILE && exit
```
###### Perform a branching conditional
```bash
true && { echo success;} || { echo failed; }
```
###### Pipe stdout and stderr to separate commands
```bash
some_command > >(/bin/cmd_for_stdout) 2> >(/bin/cmd_for_stderr)
```
###### Redirect stdout and stderr each to separate files and print both to the screen
```bash
(some_command 2>&1 1>&3 | tee errorlog ) 3>&1 1>&2 | tee stdoutlog
```
###### List of commands you use most often
```bash
history | \
awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | \
grep -v "./" | \
column -c3 -s " " -t | \
sort -nr | nl | head -n 20
```
###### Empty a file (truncate to 0 size)
```bash
>filename
```
###### Quickly backup a file
```bash
cp filename{,.orig}
```
###### Delete all files in a folder that don't match a certain file extension
```bash
rm !(*.foo|*.bar|*.baz)
```
###### Edit a file on a remote host using vim
```bash
vim scp://user@host//etc/fstab
```
###### Create a directory and change into it at the same time
```bash
mkd () { mkdir -p "$@" && cd "$@"; }
```
###### Convert uppercase files to lowercase files
```bash
rename 'y/A-Z/a-z/' *
```
###### Print a row of characters across the terminal
```bash
printf "%`tput cols`s" | tr ' ' '#'
```
###### Show shell history without line numbers
```bash
history | cut -c 8-
fc -l -n 1 | sed 's/^\s*//'
```
###### Run command(s) after exit session
```bash
cat > /etc/profile << __EOF__
_after_logout() {
username=$(whoami)
for _pid in $(ps afx | grep sshd | grep "$username" | awk '{print $1}') ; do
kill -9 $_pid
done
}
trap _after_logout EXIT
__EOF__
```
###### Generate a sequence of numbers
```bash
for ((i=1; i<=10; i+=2)) ; do echo $i ; done
# alternative: seq 1 2 10
for ((i=5; i<=10; ++i)) ; do printf '%02d\n' $i ; done
# alternative: seq -w 5 10
```
___
##### Tool: [mount](https://en.wikipedia.org/wiki/Mount_(Unix))
###### Mount a temporary ram partition
```bash
mount -t tmpfs tmpfs /mnt -o size=64M
```
* `-t` - filesystem type
* `-o` - mount options
###### Remount a filesystem as read/write
```bash
mount -o remount,rw /
```
___
##### Tool: [fuser](https://en.wikipedia.org/wiki/Fuser_(Unix))
###### Kills a process that is locking a file
```bash
fuser -k filename
```
###### Show what PID is listening on specific port
```bash
fuser -v 53/udp
```
___
##### Tool: [ps](https://en.wikipedia.org/wiki/Ps_(Unix))
###### Show a 4-way scrollable process tree with full details
```bash
ps awwfux | less -S
```
###### Processes per user counter
```bash
ps hax -o user | sort | uniq -c | sort -r
```
___
##### Tool: [find](https://en.wikipedia.org/wiki/Find_(Unix))
###### Find files that have been modified on your system in the past 60 minutes
```bash
find / -mmin 60 -type f
```
###### Find all files larger than 20M
```bash
find / -type f -size +20M
```
###### Find duplicate files (based on MD5 hash)
```bash
find -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 33
```
###### Change permission only for files
```bash
cd /var/www/site && find . -type f -exec chmod 766 {} \;
cd /var/www/site && find . -type f -exec chmod 664 {} +
```
###### Change permission only for directories
```bash
cd /var/www/site && find . -type d -exec chmod g+x {} \;
cd /var/www/site && find . -type d -exec chmod g+rwx {} +
```
###### Find files and directories for specific user
```bash
find . -user <username> -print
```
###### Find files and directories for all without specific user
```bash
find . \!-user <username> -print
```
###### Delete older files than 60 days
```bash
find . -type f -mtime +60 -delete
```
###### Recursively remove all empty sub-directories from a directory
```bash
find . -depth -type d -empty -exec rmdir {} \;
```
###### How to find all hard links to a file
```bash
find </path/to/dir> -xdev -samefile filename
```
___
##### Tool: [top](https://en.wikipedia.org/wiki/Top_(software))
###### Use top to monitor only all processes with the specific string
```bash
top -p $(pgrep -d , <str>)
```
* `<str>` - process containing str (eg. nginx, worker)
___
##### Tool: [kill](https://en.wikipedia.org/wiki/Kill_(command))
###### Kill a process running on port
```bash
kill -9 $(lsof -i :<port> | awk '{l=$2} END {print l}')
```
___
##### Tool: [diff](https://en.wikipedia.org/wiki/Diff)
###### Compare two directory trees
```bash
diff <(cd directory1 && find | sort) <(cd directory2 && find | sort)
```
___
##### Tool: [tail](https://en.wikipedia.org/wiki/Tail_(Unix))
###### Annotate tail -f with timestamps
```bash
tail -f file | while read ; do echo "$(date +%T.%N) $REPLY" ; done
```
###### Analyse an Apache access log for the most common IP addresses
```bash
tail -10000 access_log | awk '{print $1}' | sort | uniq -c | sort -n | tail
```
###### Analyse web server log and show only 5xx http codes
```bash
tail -n 100 -f /path/to/logfile | grep "HTTP/[1-2].[0-1]\" [5]"
```
___
##### Tool: [tar](https://en.wikipedia.org/wiki/Tar_(computing))
###### System backup with exclude specific directories
```bash
cd /
tar -czvpf /mnt/system$(date +%d%m%Y%s).tgz --directory=/ \
--exclude=proc/* --exclude=sys/* --exclude=dev/* --exclude=mnt/* .
```
###### System backup with exclude specific directories (pigz)
```bash
cd /
tar cvpf /backup/snapshot-$(date +%d%m%Y%s).tgz --directory=/ \
--exclude=proc/* --exclude=sys/* --exclude=dev/* \
--exclude=mnt/* --exclude=tmp/* --use-compress-program=pigz .
```
___
##### Tool: [dump](https://en.wikipedia.org/wiki/Dump_(program))
###### System backup to file
```bash
dump -y -u -f /backup/system$(date +%d%m%Y%s).lzo /
```
###### Restore system from lzo file
```bash
cd /
restore -rf /backup/system$(date +%d%m%Y%s).lzo
```
___
##### Tool: [cpulimit](http://cpulimit.sourceforge.net/)
###### Limit the cpu usage of a process
```bash
cpulimit -p pid -l 50
```
___
##### Tool: [pwdx](https://www.cyberciti.biz/faq/unix-linux-pwdx-command-examples-usage-syntax/)
###### Show current working directory of a process
```bash
pwdx <pid>
```
___
##### Tool: [taskset](https://www.cyberciti.biz/faq/taskset-cpu-affinity-command/)
###### Start a command on only one CPU core
```bash
taskset -c 0 <command>
```
___
##### Tool: [tr](https://en.wikipedia.org/wiki/Tr_(Unix))
###### Show directories in the PATH, one per line
```bash
tr : '\n' <<<$PATH
```
___
##### Tool: [chmod](https://en.wikipedia.org/wiki/Chmod)
###### Remove executable bit from all files in the current directory
```bash
chmod -R -x+X *
```
###### Restore permission for /bin/chmod
```bash
# 1:
cp /bin/ls chmod.01
cp /bin/chmod chmod.01
./chmod.01 700 file
# 2:
/bin/busybox chmod 0700 /bin/chmod
# 3:
setfacl --set u::rwx,g::---,o::--- /bin/chmod
```
___
##### Tool: [who](https://en.wikipedia.org/wiki/Who_(Unix))
###### Find last reboot time
```bash
who -b
```
___
##### Tool: [screen](https://en.wikipedia.org/wiki/GNU_Screen)
###### Start screen in detached mode
```bash
screen -d -m [<command>]
```
___
##### Tool: [du](https://en.wikipedia.org/wiki/GNU_Screen)
###### Show 20 biggest directories with 'K M G'
```bash
du | \
sort -r -n | \
awk '{split("K M G",v); s=1; while($1>1024){$1/=1024; s++} print int($1)" "v[s]"\t"$2}' | \
head -n 20
```
___
##### Tool: [inotifywait](https://en.wikipedia.org/wiki/GNU_Screen)
###### Init tool everytime a file in a directory is modified
```bash
while true ; do inotifywait -r -e MODIFY dir/ && ls dir/ ; done;
```
___
##### Tool: [openssl](https://www.openssl.org/)
###### Testing connection to remote host
```bash
echo | openssl s_client -connect google.com:443 -showcerts
```
###### Testing connection to remote host (with SNI support)
```bash
echo | openssl s_client -showcerts -servername google.com -connect google.com:443
```
###### Testing connection to remote host with specific ssl version
```bash
openssl s_client -tls1_2 -connect google.com:443
```
###### Testing connection to remote host with specific ssl cipher
```bash
openssl s_client -cipher 'AES128-SHA' -connect google.com:443
```
###### Generate private key
```bash
# _ciph: des3, aes
( _ciph="des3" ; _fd="private.key" ; _len="2048" ; \
openssl genrsa -${_ciph} -out ${_fd} ${_len} )
```
###### Remove password from private key
```bash
( _fd="private.key" ; _fd_unp="private_unp.key" ; \
openssl rsa -in ${_fd} -out ${_fd_unp} )
```
###### Get public key from private key
```bash
( _fd="private.key" ; _fd_pub="public.key" ; \
openssl rsa -pubout -in ${_fd} -out ${_fd_pub} )
```
###### Generate private key + csr
```bash
( _fd="private.key" ; _fd_csr="request.csr" ; _len="2048" ; \
openssl req -out ${_fd_csr} -new -newkey rsa:${_len} -nodes -keyout ${_fd} )
```
###### Generate csr
```bash
( _fd="private.key" ; _fd_csr="request.csr" ; \
openssl req -out ${_fd_csr} -new -key ${_fd} )
```
###### Generate csr (metadata from exist certificate)
```bash
( _fd="private.key" ; _fd_csr="request.csr" ; _fd_crt="cert.crt" ; \
openssl x509 -x509toreq -in ${_fd_crt} -out ${_fd_csr} -signkey ${_fd} )
```
###### Generate csr with -config param
```bash
( _fd="private.key" ; _fd_csr="request.csr" ; \
openssl req -new -sha256 -key ${_fd} -out ${_fd_csr} \
-config <(
cat <<-EOF
[req]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn
[ dn ]
C=<two-letter ISO abbreviation for your country>
ST=<state or province where your organization is legally located>
L=<city where your organization is legally located>
O=<legal name of your organization>
OU=<section of the organization>
CN=<fully qualified domain name>
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = <fully qualified domain name>
DNS.2 = <next domain>
DNS.3 = <next domain>
EOF
))
```
###### Convert DER to PEM
```bash
( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
openssl x509 -in ${_fd_der} -inform der -outform pem -out ${_fd_pem} )
```
###### Convert PEM to DER
```bash
( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
openssl x509 -in ${_fd_pem} -outform der -out ${_fd_der} )
```
###### Checking whether the private key and the certificate match
```bash
(openssl rsa -noout -modulus -in private.key | openssl md5 ; openssl x509 -noout -modulus -in certificate.crt | openssl md5) | uniq
```
___
##### Tool: [gnutls-cli](https://gnutls.org/manual/html_node/gnutls_002dcli-Invocation.html)
###### Testing connection to remote host (with sni)
```bash
gnutls-cli -p 443 google.com
```
###### Testing connection to remote host (without sni)
```bash
gnutls-cli --disable-sni -p 443 google.com
```
___
##### Tool: [secure-delete](https://wiki.archlinux.org/index.php/Securely_wipe_disk)
###### Secure delete with shred
```bash
shred -vfuz -n 10 file
shred --verbose --random-source=/dev/urandom -n 1 /dev/sda
```
###### Secure delete with scrub
```bash
scrub -p dod /dev/sda
scrub -p dod -r file
```
###### Secure delete with badblocks
```bash
badblocks -s -w -t random -v /dev/sda
badblocks -c 10240 -s -w -t random -v /dev/sda
```
###### Secure delete with secure-delete
```bash
srm -vz /tmp/file
sfill -vz /local
sdmem -v
swapoff /dev/sda5 && sswap -vz /dev/sda5
```
___
##### Tool: [dd](https://en.wikipedia.org/wiki/Dd_(Unix))
###### Show dd status every so often
```bash
dd <dd_params> status=progress
watch --interval 5 killall -USR1 dd
```
<a name="http-https"><b>HTTP/HTTPS</b></a>
##### Tool: [curl](https://curl.haxx.se)
```bash
curl -Iks https://www.google.com
```
* `-I` - show response headers only
* `-k` - insecure connection when using ssl
* `-s` - silent mode (not display body)
```bash
curl -Iks --location -X GET -A "x-agent" https://www.google.com
```
* `--location` - follow redirects
* `-X` - set method
* `-A` - set user-agent
```bash
curl -Iks --location -X GET -A "x-agent" --proxy http://127.0.0.1:16379 https://www.google.com
```
* `--proxy [socks5://|http://]` - set proxy server
###### Check DNS and HTTP trace with headers for specific domains
```bash
### Set domains and external dns servers.
_domain_list=(google.com) ; _dns_list=("8.8.8.8" "1.1.1.1")
for _domain in "${_domain_list[@]}" ; do
printf '=%.0s' {1..48}
echo
printf "[\\e[1;32m+\\e[m] resolve: %s\\n" "$_domain"
for _dns in "${_dns_list[@]}" ; do
# Resolve domain.
host "${_domain}" "${_dns}"
echo
done
for _proto in http https ; do
printf "[\\e[1;32m+\\e[m] trace + headers: %s://%s\\n" "$_proto" "$_domain"
# Get trace and http headers.
curl -Iks -A "x-agent" --location "${_proto}://${_domain}"
echo
done
done
unset _domain_list _dns_list
```
___
##### Tool: [httpie](https://httpie.org/)
```bash
http -p Hh https://www.google.com
```
* `-p` - print request and response headers
* `H` - request headers
* `B` - request body
* `h` - response headers
* `b` - response body
```bash
http -p Hh --follow --max-redirects 5 --verify no https://www.google.com
```
* `-F, --follow` - follow redirects
* `--max-redirects N` - maximum for `--follow`
* `--verify no` - skip SSL verification
```bash
http -p Hh --follow --max-redirects 5 --verify no --proxy http:http://127.0.0.1:16379 https://www.google.com
```
* `--proxy [http:]` - set proxy server
<a name="network"><b>Network</b></a>
##### Tool: [ssh](https://www.openssh.com/)
###### Compare a remote file with a local file
```bash
ssh user@host cat /path/to/remotefile | diff /path/to/localfile -
```
###### SSH connection through host in the middle
```bash
ssh -t reachable_host ssh unreachable_host
```
###### Run command over ssh on remote host
```bash
cat > cmd.txt << __EOF__
cat /etc/hosts
__EOF__
ssh host -l user $(<cmd.txt)
```
###### Get public key from private key
```bash
ssh-keygen -y -f ~/.ssh/id_rsa
```
###### Get all fingerprints
```bash
ssh-keygen -l -f .ssh/known_hosts
```
###### Ssh authentication with user password
```bash
ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@remote_host
```
###### Ssh authentication with publickey
```bash
ssh -o PreferredAuthentications=publickey -o PubkeyAuthentication=yes -i id_rsa user@remote_host
```
###### Simple recording SSH session
```bash
function _ssh_sesslog() {
_sesdir="<path/to/session/logs>"
mkdir -p "${_sesdir}" && \
ssh $@ 2>&1 | tee -a "${_sesdir}/$(date +%Y%m%d).log"
}
# Alias:
alias ssh='_ssh_sesslog'
```
###### Using Keychain for SSH logins
```bash
### Delete all of ssh-agent's keys.
function _scl() {
/usr/bin/keychain --clear
}
### Add key to keychain.
function _scg() {
/usr/bin/keychain /path/to/private-key
source "$HOME/.keychain/$HOSTNAME-sh"
}
```
___
##### Tool: [linux-dev](https://www.tldp.org/LDP/abs/html/devref1.html)
###### Testing remote connection to port
```bash
timeout 1 bash -c "</dev/<proto>/<host>/<port>" >/dev/null 2>&1 ; echo $?
```
* `<proto` - set protocol (tcp/udp)
* `<host>` - set remote host
* `<port>` - set destination port
###### Read and write to TCP or UDP sockets with common bash tools
```bash
exec 5<>/dev/tcp/<host>/<port>; cat <&5 & cat >&5; exec 5>&-
```
___
##### Tool: [tcpdump](http://www.tcpdump.org/)
```bash
tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443
```
* `-n` - don't convert addresses
* `-e` - print the link-level headers
* `-i [iface]` - set interface
* `-Q|-D [in|out|inout]` - choose send/receive direction (`-D` - for old tcpdump versions)
* `host [ip|hostname]` - set host, also `[host not]`
* `[and|or]` - set logic
* `port [1-65535]` - set port number, also `[port not]`
```bash
tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443 -c 5 -w tcpdump.pcap
```
* `-c [num]` - capture only num number of packets
* `-w [filename]` - write packets to file, `-r [filename]` - reading from file
___
##### Tool: [tcpick](http://tcpick.sourceforge.net/)
###### Analyse packets in real-time
```bash
while true ; do tcpick -a -C -r dump.pcap ; sleep 2 ; clear ; done
```
___
##### Tool: [ngrep](http://ngrep.sourceforge.net/usage.html)
```bash
ngrep -d eth0 "www.google.com" port 443
```
* `-d [iface|any]` - set interface
* `[domain]` - set hostname
* `port [1-65535]` - set port number
```bash
ngrep -d eth0 "www.google.com" (host 10.240.20.2) and (port 443)
```
* `(host [ip|hostname])` - filter by ip or hostname
* `(port [1-65535])` - filter by port number
```bash
ngrep -d eth0 -qt -O ngrep.pcap "www.google.com" port 443
```
* `-q` - quiet mode (only payloads)
* `-t` - added timestamps
* `-O [filename]` - save output to file, `-I [filename]` - reading from file
```bash
ngrep -d eth0 -qt 'HTTP' 'tcp'
```
* `HTTP` - show http headers
* `tcp|udp` - set protocol
* `[src|dst] host [ip|hostname]` - set direction for specific node
```bash
ngrep -l -q -d eth0 -i "User-Agent: curl*"
```
* `-l` - stdout line buffered
* `-i` - case-insensitive search
___
##### Tool: [hping3](http://www.hping.org/)
```bash
hping3 -V -p 80 -s 5050 <scan_type> www.google.com
```
* `-V|--verbose` - verbose mode
* `-p|--destport` - set destination port
* `-s|--baseport` - set source port
* `<scan_type>` - set scan type
* `-F|--fin` - set FIN flag, port open if no reply
* `-S|--syn` - set SYN flag
* `-P|--push` - set PUSH flag
* `-A|--ack` - set ACK flag (use when ping is blocked, RST response back if the port is open)
* `-U|--urg` - set URG flag
* `-Y|--ymas` - set Y unused flag (0x80 - nullscan), port open if no reply
* `-M 0 -UPF` - set TCP sequence number and scan type (URG+PUSH+FIN), port open if no reply
```bash
hping3 -V -c 1 -1 -C 8 www.google.com
```
* `-c [num]` - packet count
* `-1` - set ICMP mode
* `-C|--icmptype [icmp-num]` - set icmp type (default icmp-echo = 8)
```bash
hping3 -V -c 1000000 -d 120 -S -w 64 -p 80 --flood --rand-source <remote_host>
```
* `--flood` - sent packets as fast as possible (don't show replies)
* `--rand-source` - random source address mode
* `-d --data` - data size
* `-w|--win` - winsize (default 64)
___
##### Tool: [netcat](http://netcat.sourceforge.net/)
```bash
nc -kl 5000
```
* `-l` - listen for an incoming connection
* `-k` - listening after client has disconnected
* `>filename.out` - save receive data to file (optional)
```bash
nc 192.168.0.1 5051 < filename.in
```
* `< filename.in` - send data to remote host
```bash
nc -vz 10.240.30.3 5000
```
* `-v` - verbose output
* `-z` - scan for listening daemons
```bash
nc -vzu 10.240.30.3 1-65535
```
* `-u` - scan only udp ports
###### Transfer data file (archive)
```bash
server> nc -l 5000 | tar xzvfp -
client> tar czvfp - /path/to/dir | nc 10.240.30.3 5000
```
###### Launch remote shell
```bash
server> nc -l 5000 -e /bin/bash
client> nc 10.240.30.3 5000
```
###### Simple file server
```bash
while true ; do nc -l 5000 | tar -xvf - ; done
```
###### Simple minimal HTTP Server
```bash
while true ; do nc -l -p 1500 -c 'echo -e "HTTP/1.1 200 OK\n\n $(date)"' ; done
```
###### Simple HTTP Server
> Restarts web server after each request - remove `while` condition for only single connection.
```bash
cat > index.html << __EOF__
<!doctype html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title></title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<p>
Hello! It's a site.
</p>
</body>
</html>
__EOF__
```
```bash
server> while : ; do \
(echo -ne "HTTP/1.1 200 OK\r\nContent-Length: $(wc -c <index.html)\r\n\r\n" ; cat index.html;) | \
nc -l -p 5000 \
; done
```
* `-p` - port number
###### Simple HTTP Proxy (single connection)
```bash
#!/usr/bin/env bash
if [[ $# != 2 ]] ; then
printf "%s\\n" \
"usage: ./nc-proxy listen-port bk_host:bk_port"
fi
_listen_port="$1"
_bk_host=$(echo "$2" | cut -d ":" -f1)
_bk_port=$(echo "$2" | cut -d ":" -f2)
printf " lport: %s\\nbk_host: %s\\nbk_port: %s\\n\\n" \
"$_listen_port" "$_bk_host" "$_bk_port"
_tmp=$(mktemp -d)
_back="$_tmp/pipe.back"
_sent="$_tmp/pipe.sent"
_recv="$_tmp/pipe.recv"
trap 'rm -rf "$_tmp"' EXIT
mkfifo -m 0600 "$_back" "$_sent" "$_recv"
sed "s/^/=> /" <"$_sent" &
sed "s/^/<= /" <"$_recv" &
nc -l -p "$_listen_port" <"$_back" | \
tee "$_sent" | \
nc "$_bk_host" "$_bk_port" | \
tee "$_recv" >"$_back"
```
```bash
server> chmod +x nc-proxy && ./nc-proxy 8080 192.168.252.10:8000
lport: 8080
bk_host: 192.168.252.10
bk_port: 8000
client> http -p h 10.240.30.3:8080
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=31536000
Content-Length: 2748
Content-Type: text/html; charset=utf-8
Date: Sun, 01 Jul 2018 20:12:08 GMT
Last-Modified: Sun, 01 Apr 2018 21:53:37 GMT
```
###### Create a single-use TCP or UDP proxy
```bash
### TCP -> TCP
nc -l -p 2000 -c "nc [ip|hostname] 3000"
### TCP -> UDP
nc -l -p 2000 -c "nc -u [ip|hostname] 3000"
### UDP -> UDP
nc -l -u -p 2000 -c "nc -u [ip|hostname] 3000"
### UDP -> TCP
nc -l -u -p 2000 -c "nc [ip|hostname] 3000"
```
___
##### Tool: [socat](http://www.dest-unreach.org/socat/doc/socat.html/)
###### Testing remote connection to port
```bash
socat - TCP4:10.240.30.3:22
```
* `-` - standard input (STDIO)
* `TCP4:<params>` - set tcp4 connection with specific params
* `[hostname|ip]` - set hostname/ip
* `[1-65535]` - set port number
###### Redirecting TCP-traffic to a UNIX domain socket under Linux
```bash
socat TCP-LISTEN:1234,bind=127.0.0.1,reuseaddr,fork,su=nobody,range=127.0.0.0/8 UNIX-CLIENT:/tmp/foo
```
* `TCP-LISTEN:<params>` - set tcp listen with specific params
* `[1-65535]` - set port number
* `bind=[hostname|ip]` - set bind hostname/ip
* `reuseaddr` - allows other sockets to bind to an address
* `fork` - keeps the parent process attempting to produce more connections
* `su=nobody` - set user
* `range=[ip-range]` - ip range
* `UNIX-CLIENT:<params>` - communicates with the specified peer socket
* `filename` - define socket
___
##### Tool: [lsof](https://en.wikipedia.org/wiki/Lsof)
###### Show process that use internet connection at the moment
```bash
lsof -P -i -n
```
###### Show process that use specific port number
```bash
lsof -i tcp:443
```
###### Lists all listening ports together with the PID of the associated process
```bash
lsof -Pan -i tcp -i udp
```
###### List all open ports and their owning executables
```bash
lsof -i -P | grep -i "listen"
```
###### Show all open ports
```bash
lsof -Pnl -i
```
###### Show open ports (LISTEN)
```bash
lsof -Pni4 | grep LISTEN | column -t
```
###### List all files opened by a particular command
```bash
lsof -c "process"
```
###### View user activity per directory
```bash
lsof -u username -a +D /etc
```
###### Show 10 Largest Open Files
```bash
lsof / | \
awk '{ if($7 > 1048576) print $7/1048576 "MB" " " $9 " " $1 }' | \
sort -n -u | tail | column -t
```
___
##### Tool: [netstat](https://en.wikipedia.org/wiki/Netstat)
###### Graph # of connections for each hosts
```bash
netstat -an | \
grep ESTABLISHED | \
awk '{print $5}' | \
awk -F: '{print $1}' | \
grep -v -e '^[[:space:]]*$' | \
sort | uniq -c | \
awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'
```
###### Monitor open connections for specific port including listen, count and sort it per IP
```bash
watch "netstat -plan | grep :443 | awk {'print \$5'} | cut -d: -f 1 | sort | uniq -c | sort -nk 1"
```
___
##### Tool: [rsync](https://en.wikipedia.org/wiki/Rsync)
###### Rsync remote data as root using sudo
```bash
rsync --rsync-path 'sudo rsync' username@hostname:/path/to/dir/ /local/
```
___
##### Tool: [host](https://en.wikipedia.org/wiki/Host_(Unix))
###### Resolves the domain name (using external dns server)
```bash
host google.com 9.9.9.9
```
###### Checks the domain administrator (SOA record)
```bash
host -t soa google.com 9.9.9.9
```
___
##### Tool: [dig](https://en.wikipedia.org/wiki/Dig_(command))
###### Resolves the domain name (short output)
```bash
dig google.com +short
```
###### Lookup NS record for specific domain
```bash
dig @9.9.9.9 google.com NS
```
###### Query only answer section
```bash
dig google.com +nocomments +noquestion +noauthority +noadditional +nostats
```
###### Query ALL DNS Records
```bash
dig google.com ANY +noall +answer
```
###### DNS Reverse Look-up
```bash
dig -x 172.217.16.14 +short
```
___
##### Tool: [network-other](https://github.com/trimstray/awesome-ninja-admins#tool-network-other)
###### Get all subnets for specific AS (Autonomous system)
```bash
AS="AS32934"
whois -h whois.radb.net -- "-i origin ${AS}" | \
grep "^route:" | \
cut -d ":" -f2 | \
sed -e 's/^[ \t]//' | \
sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 | \
cut -d ":" -f2 | \
sed -e 's/^[ \t]/allow /' | \
sed 's/$/;/' | \
sed 's/allow */subnet -> /g'
```
###### Resolves domain name from dns.google.com with curl and jq
```bash
_dname="google.com" ; curl -s "https://dns.google.com/resolve?name=${_dname}&type=A" | jq .
```
<a name="programming"><b>Programming</b></a>
##### Tool: [awk](http://www.grymoire.com/Unix/Awk.html)
###### Remove duplicate entries in a file without sorting
```bash
awk '!x[$0]++' filename
```
###### Exclude multiple columns using AWK
```bash
awk '{$1=$3=""}1' filename
```
___
##### Tool: [sed](http://www.grymoire.com/Unix/Sed.html)
###### To print a specific line from a file
```bash
sed -n 10p /path/to/file
```
###### Remove a specific line from a file
```bash
sed -i 10d /path/to/file
# alternative (BSD): sed -i'' 10d /path/to/file
```
###### Remove a range of lines from a file
```bash
sed -i <file> -re '<start>,<end>d'
```
___
##### Tool: [grep](http://www.grymoire.com/Unix/Grep.html)
###### Search for a "pattern" inside all files in the current directory
```bash
grep -rn "pattern"
grep -RnisI "pattern" *
fgrep "pattern" * -R
```
###### Remove blank lines from a file and save output to new file
```bash
grep . filename > newfilename
```
###### Except multiple patterns
```bash
grep -vE '(error|critical|warning)' filename
```
###### Show data from file without comments
```bash
grep -v ^[[:space:]]*# filename
```
###### Show data from file without comments and new lines
```bash
egrep -v '#|^$' filename
```