On the Attack-Defense CTF last season we were given 12 hosts for attack and 20 minutes for primary exploration. Start nmap piece by piece – burn half the time on the wait. A series of six lines of Bash with background processes through & and wait for synchronization laid scanning of all 12 goals for one and a half minutes. Bash is not faster than specialized scanners – it just works on any Unix machine with a terminal, without installing packages. In the terminology of MITRE ATT&CK it Unix Shell (T1059.004, Execution Tactics) – one of the basic methods of executing commands on Unix systems, which in the CTF permeates the entire chain from intelligence to collecting flags. Below are ready-made scripts and single-liners that close the three main routines of the competition: ports, scanner output parsing and mass file processing. Each is dismantled in line, describing the rake on which I came myself.
Requirements for the environment
[Applicable: CTF jeopardy, CTF attack-defense, internal pentest on Linux infrastructure]
Before copying scripts – make sure the surroundings don’t throw up surprises.
OS: Kali Linux, Parrot OS, Ubuntu 20.04+, macOS with reservations. WSL2 on Windows 10/11 works, but adds a network layer — /dev/tcp and ping may behave differently
Bash: 4.x and above version. Verification – bash --version. On macOS from the box stands Bash 3.2 (the license GPLv3 prevents Apple from updating), put the relevant through brew install bash
RAM: 512 MB is enough for all scripts from the article. With mass background overtaking through & for hundreds of processes - 2 GB minimum
Dependencies: curl, grep, awk, sed – is in any distribution out of the box. nmap and jq placed through a package manager: apt install nmap jq
Editor: nano for beginners, vim for those who spent an hour studying :wq. Each script begins with a shebang #!/bin/bash and becomes executed after chmod +x script.sh. Without chmod +x get Permission denied The first trap that everyone goes through
Create a working directory immediately: mkdir -p ~/ctf/scripts && cd ~/ctf/scripts. All the weekend script files will fall into one place and you won't lose results in the middle of the competition.
Overtaking bash ports : from /dev/tcp to nmap wrapper
Works if: Bash is compiled with support /dev/tcp (Kali, Ubuntu, Parrot — by default yes), the target host is available by TCP. Does not work if: used dash or sh instead of bash ( in Debian /bin/sh refers to dash, where /dev/tcp missing), Bash collected without --enable-net-redirections, Target behind the firewall with DROP policy.
When nmap unavailable — and on CTF infrastructure with tool restrictions, it happens regularly — Bash is able to check TCP ports through a virtual device /dev/tcp. This is not a file on the disk, but a built-in shell mechanism: writing in /dev/tcp/host/port initiates a TCP connection. Port open - return code 0, closed - error.
#!/bin/bash
HOST="$1"
for port in $(seq 1 1024); do
(echo >/dev/tcp/"$HOST"/"$port") 2>/dev/null && \
echo "[+] $HOST:$port open"
done
Script accepts IP first argument ($1). Cycle seq 1 1024 Searches the first thousand ports. Construction echo >/dev/tcp/"$HOST"/"$port" tries to open TCP connection, brackets () create subshell (isolate the error). Redirect 2>/dev/null suppresses reports of closed ports. Operator && performs echo only with a successful connection.
The problem is the script is consistent. At 1024 ports with a default timeout (about a second to a closed port) it will take up to 17 minutes. On the CTF it is eternity. The solution is parallelization. Add & after echo in the body of the cycle, and after done put wait. Ampersand sends each check to the background process, wait Waiting for the completion of all. For 1024 ports, time drops to 3-5 seconds.
But here is the second trap: 1024 simultaneous processes the machine will survive, and 65535 - most likely not. Limit the parallelism: xargs -P 50 -I{} bash -c 'echo >/dev/tcp/"$1"/{} 2>/dev/null && echo "[+] $1:{} open"' _ "$HOST" starts a maximum of 50 checks at a time. Or GNU parallel with the key -j 50.
When /dev/tcp does not save
UDP Ports: /dev/tcp works only with TCP. For UDP you need nmap -sU or nc -u -w 1 $HOST $port
Definition of services: Clean Bash does not send banner requests. You will learn that port 22 is open, but you will not define the OpenSH version. For versions — nmap -sV or manual nc -w 2 $HOST $port < /dev/null
Timeouts on DROP firewalls: if the firewall drops the packets (does not respond to the RST), each closed port hangs until the timeout. There is no built-in timeout for Bash /dev/tcp – can be wrapped through timeout 1 bash -c "echo >/dev/tcp/$HOST/$port", but it slows down execution and produces daughter processes
Detection: any successive port overtake is detected elementary. In SigmaHQ 19 rules for T1059.004 (Unix Shell), including lnx_shell_susp_commands.yml. Port Scan category (#14 according to the AbuseIPDB classification) is a standard trigger for automatic locks. On CTF it is usually not a problem, on a real pentest - consider
Parsing withdrawal commands linux: grep, awk and sed in combat
Works if: GNU grep is installed (Kali, Ubuntu – default), file in text format. Does not work if: used BSD grep without -P (macOS - replace with grep -oE), binary file (use strings before grep).
The main task in the CTF is not to run the scanner, but to quickly pull the desired one out of its output. Russian-speaking guides usually end on grep "open" – it covers the percentage of ten real tasks of parsing. Let's analyze the processing of the output nmap and curl – two tools that generate the main flow of text data at competitions.
Disassembly nmap -oG via grep and awk
The key flag — -oG (grepable output). Format nmap -oG gives one line to the host with all the open ports – ideal for conveyor processing. Standard multi-line conclusion nmap parsing is inconvenient – each port on a separate line, between hosts empty lines and decorative frames.
Launch: nmap -sC -sV -oG scan.grep -oN scan.txt 10.10.0.0/24. Flag -oG saves the greepable format, -oN – normal (for reading with eyes). Two formats at the same time is a habit that saves time on CTF. I always do that.
Extract open port 80 IP addresses: grep '80/open' scan.grep | awk '{print $2}'. Here awk '{print $2}' outputs the second line field — IP address (first field in format -oG – word Host
.
Get a list of all open ports of a particular host: grep '10.10.0.5' scan.grep | grep -oP '\d+/open' | cut -d'/' -f1. Regularity \d+/open through grep -oP (Perl-compatible regulars) extracts the port number before /open, eh cut -d'/' -f1 cuts everything off after the slash. On macOS -P not supported – replace with grep -oE '[0-9]+/open'.
Script for mass processing of greepable-output:
#!/bin/bash
nmap -oG, выводит ip
grep '/open/' "$1" | while read -r line; do
ip=$(echo "$line" | awk '{print $2}')
ports=$(echo "$line" | grep -oP '\d+/open' | \
cut -d'/' -f1 | tr '\n' ',')
echo "$ip -> $ports"
done
Accepts the file nmap -oG The first argument. For each line with open ports, IP is extracted via awk and forms a compact list of ports. Team tr '\n' ',' replaces translations of lines with commas - the output of the view is obtained 10.10.0.5 -> 22,80,443,.
A few patterns sed, which on the CTF are constantly useful. Removing blank lines: sed '/^$/d' output.txt. Extraction of value after password=: sed -n 's/.*password[=:]\s*\([^ ]*\).*/\1/p' config.txt. Break the line hash:salt:user on separate lines: echo "hash:salt:user" | sed 's/:/\n/g'. Output lines from 10 to 20 from the log: sed -n '10,20p' access.log.
Extract data from HTTP responses
On CTF, you often need to process dozens of HTTP answers: find a different length, pull the token out of the title, filter out answers with a certain status.
Basic pattern – curl -s -o /dev/null -w "%{http_code} %{size_download}" – returns HTTP code and response size in bytes without body. Turning into a cycle with a wordlist, you get a simple fuzzer directories:
while read -r dir; do code=$(curl -s -o /dev/null -w "%{http_code}" "http://target/$dir"); [ "$code" != "404" ] && echo "$code $dir"; done < wordlist.txt
Square Brackets in [ "$code" != "404" ] – call the team test. Gaps around brackets and operator are mandatory. Recording ["$code" without space, and hello, [: command not found. The most annoying syntactic feature of Bash and the most frequent question from beginners on forums.
For JSON replies — jq: curl -s http://target/api/user | jq -r '.token'. Without jq can be through grep -oP '"token":"[^"]+"' | cut -d'"' -f4, but it’s fragile – regularly breaks down on nested quotes. For the headlines: curl -sI http://target | grep -i 'set-cookie' – flag -I requests only headings (HEAD request).
Mass processing of bash files: hunting for flags
Works if: has read-access to target directories, GNU grep supports -r. Does not work if: file system encrypted, files in non-standard encoding (UTF-16 — grep will not see ASCII strings without prior conversion through iconv).
In MITRE ATT&CK it is the intersection of two techniques: File and Directory Discovery (T1083, Discovery Tactics) – transfer of files through find and ls – and Automated Collection (T1119, Collection tactics) – automated data collection from found files. In CTF context: you got shell and are looking for a flag, or downloaded the file system dump and processing it locally.
Recursive search by file system
Classic single-line from CTF-chit-shits (HackTricks, PayloadsAllTheThings): grep -rE 'flag\{|CTF\{|HTB\{|THM\{' / 2>/dev/null. Flag -r – recursive search, -E – extended regulars, 2>/dev/null – suppression of mistakes Permission denied. Problem: On a live system with thousands of files, it's slow and generates noise in the logs.
The optimized option is to limit the search area to typical CTF directories:
#!/bin/bash
DIRS="/root /home /opt /var/www /tmp /srv"
PATTERN='flag\{|CTF\{|HTB\{|THM\{|picoCTF\{'
for d in $DIRS; do
[ -d "$d" ] && grep -rlE "$PATTERN" "$d" 2>/dev/null
done
Checking [ -d "$d" ] excludes non-existent ways (without it grep will work, but will issue an error). Flag -l displays only file names with matches, without the lines themselves - faster and more compact. Variable DIRS – typical places where CTF platforms hide flags: home directories of users, /opt (Custom applications), /var/www (Web root) /tmp (temporary files).
For binary files grep Default outputs Binary file matches. Add the flag -a (process as text) or better: strings "$file" | grep -E "$PATTERN" – utility strings extracts printed sequences from binary, and further grep works with clean text. Team find / -type f -exec file {} + | grep 'ELF' find all the ELF binari, and strings with grep will handle each.
Batch processing of dumps and archives
On forensic tasks, a directory with hundreds of files of an unknown type is often found: dumps, archives, images. The task is to process everyone automatically.
Pattern with find and -exec: find ./dump -type f -name '*.gz' -exec gunzip {} \; Unpacks everything .gz-files. But on the CTF extension regularly lie – file image.png may be a ZIP archive. Trust magic bytes, not extension:
find ./dump -type f | while read -r f; do file "$f" | grep -q 'gzip' && gunzip "$f"; done
Team file "$f" determine the type by magic bytes (the first bytes of the file), grep -q 'gzip' checks the result, and only then the unpacking is started.
Critical trap: file names with spaces. Construction while read -r f breaks the default input by spaces and line translations. File my flag.txt it will become two arguments: my and flag.txt. Solution: find ... -print0 | while IFS= read -r -d '' f – zero byte separator instead of translating the line. This is one of the most insidious mistakes – the script silently passes files with spaces, and you lose the flag without knowing it. I lost half an hour on this one.
To determine the distribution of file types in the directory: find ./dump -type f -exec file {} + | awk -F: '{print $2}' | sort | uniq -c | sort -rn. Shows how much PNG, ELF, text, gzip — and helps to understand what to grab first.
Bash one-liner for hacking: working single-liners
Single-liners are a working tool of CTF-cate. Each solves a specific problem and is placed in one line of the terminal. Copied, set the goal, launched.
Search for SUID binarys to escalate privileges: find / -perm -4000 -type f 2>/dev/null. Flag -perm -4000 searching for files with SUID-bit installed. The result is checked by GTFOBins (gtfobins.github.io) — Unix-binary catalog for abuse in post-exploitation. From frequent finds: awk, bash, base64, cat – everyone can read files or give shell if there is a SUID.
System intelligence one team (T1082, System Information Discovery): echo "=== OS ==="; uname -a; echo "=== Users ==="; cat /etc/passwd | grep -v nologin; echo "=== Net ==="; ip a; echo "=== Procs ==="; ps aux --forest. Six commands in one line, instead of six separate runs.
Overtaking subdomains: while read -r sub; do host "$sub.target.com" | grep -v 'NXDOMAIN' && echo "[+] $sub"; done < subdomains.txt. Team host Resolvit DNS, grep -v 'NXDOMAIN' filters non-existent.
Generation of numerical wordlist: for i in $(seq -w 0000 9999); do echo $i; done > pins.txt. Flag -w in seq adds the leading zeros – without it 0001 generated as 1, and the four-digit PIN does not match the format.
Page Change Monitoring: watch -n 5 'curl -s http://target/score | md5sum'. Shows MD5 hash pages every 5 seconds – the hash has changed, so the contents have been updated.
Decoding Base64: cat encoded.txt | base64 -d. For URL-safe Base64 (where + replaced by -, eh / on _: cat encoded.txt | tr '_-' '/+' | base64 -d.
Writing scripts for network scanning: wrapper-pattern
Automation of CTF in practice is not a monolithic script for 200 lines, but a set of wrappers. Each turns a specific tool and transmits the result to the following. Pattern: input data — from arguments or file, the result — to the file for the next script.
#!/bin/bash
set -euo pipefail
TARGET="$1"
OUT="./results/$TARGET"
mkdir -p "$OUT"
echo "[$(date +%H:%M:%S)] Scanning $TARGET..."
nmap -sC -sV -oG "$OUT/nmap.grep" -oN "$OUT/nmap.txt" "$TARGET"
echo "[$(date +%H:%M:%S)] Extracting ports..."
grep '/open/' "$OUT/nmap.grep" | grep -oP '\d+/open' | \
cut -d'/' -f1 > "$OUT/ports.txt"
echo "[*] Found $(wc -l < "$OUT/ports.txt") open ports"
The script creates a results directory for each host, triggers nmap with two output formats, extracts open ports into a separate file. Team wc -l < "$OUT/ports.txt" counts the lines — redirection through < instead of cat file | wc -l Avoids the output of the file name as a result.
The line set -euo pipefail – three protective flags, and everyone saved me scripts more than once. set -e – interruption at the first error (without it the script will continue after the fallen nmap and the next step will get an empty file). set -u – error when referring to an uninitialized variable (typo $TAGET instead of $TARGET silently sets up an empty line, and nmap scan localhost – a pleasant surprise). set -o pipefail – the conveyor returns the error code of the last fallen command, not the last one in the chain.
Further plug in modular scripts: web_enum.sh Launches gobuster dir the HTTP Ports from ports.txt, grab_banners.sh collects banners through nc. Everyone reads a file from the results directory and writes there. Verification of arguments at the beginning of each script: [ -z "$1" ] && echo "Usage: $0 <target>" && exit 1 – without this, the script will start with an empty variable and give an incomprehensible error on the third line.
Temporary labels through echo "[$(date +%H:%M:%S)]" - not the decoration. On a CTF with limited time, they help to understand what step the script is stuck in if the process is hovering. The difference between “script works” and “script hovered on nmap for 8 minutes” is one line with date.
When bash scripts for pentest are not enough
Bash is a glue between utilities, not a full-fledged programming language. And knowing its boundaries saves hours.
Multithreading with control. Background processes through & – rough mechanism. There is no pool of flows, no task queue. For the selection of 65535 ports with a limit of 50 simultaneous connections - xargs -P 50 or GNU parallel, but logic quickly turns into porridge. In Python concurrent.futures.ThreadPoolExecutor solves this in three lines with full control.
Work with JSON/XML. jq covers a simple JSON, but parsing nested structures through chains grep | awk | sed – the path to the fragile code that breaks down every time the format changes. REST API of modern CTF platforms return JSON - you can process them on Bash, but it hurts. Python with json and requests - more reliable.
Complex logic with the state. HTTP session (cookies, CSRF tokens), repeats in errors, redirect chains - the number of Bash strings is growing exponentially. requests.Session() in Python makes it more elegant.
Cryptography. Crypto-tasks (XOR, AES, RSA) through openssl and xxd in Bash – it is possible, but impractical for something more difficult Base64. Python with pycryptodome – standard for crypto on CTF.
Rule for decision: The script has outgrown 30 lines and contains more than three nested if/while – rewrite in Python. Bash is good for a 5-15 line pipeline, where each line is a utility call through a pipeline. It’s more complicated – not its territory.
In terms of detection: Protective systems track suspicious shell activity. In SigmaHQ 19 of the Detection Rules for T1059.004 (Unix Shell), including lnx_shell_susp_rev_shells.yml to detect the reverse shell and lnx_shell_susp_commands.yml for suspicious teams. Countermeasures on D3FEND include Executable Denyllisting (D3-EDL) – whitelist execution blocking – and Content Filtering (D3-CF). On the CTF, these mechanisms are usually switched off, on the real Bash script pentest can be blocked by EDR before the first execution.
At each competition I see the same picture: teams are divided into those who write scripts for the task in minutes, and those who spend these minutes on manual input. The difference in the final score is not in knowing the vulnerabilities, but in the speed of information processing. Bash covers 80% of the routine on the CTF: ports, parsing output, search for flags on the file system. The remaining 20% — complex logic, crypto, work with API — go Python. But the skill to assemble the conveyor from grep | awk | sort | uniq in 30 seconds, while the opponent reads the manual - does not appear from reading articles. It appears after the 50-th task when you start to think of conveyors. After 200, you see the patterns in the output before you start grep. If you go to OSCP and need intelligence and scripting training, WAPT covers this in the first modules with labs for each case.
Requirements for the environment
[Applicable: CTF jeopardy, CTF attack-defense, internal pentest on Linux infrastructure]
Before copying scripts – make sure the surroundings don’t throw up surprises.
OS: Kali Linux, Parrot OS, Ubuntu 20.04+, macOS with reservations. WSL2 on Windows 10/11 works, but adds a network layer — /dev/tcp and ping may behave differently
Bash: 4.x and above version. Verification – bash --version. On macOS from the box stands Bash 3.2 (the license GPLv3 prevents Apple from updating), put the relevant through brew install bash
RAM: 512 MB is enough for all scripts from the article. With mass background overtaking through & for hundreds of processes - 2 GB minimum
Dependencies: curl, grep, awk, sed – is in any distribution out of the box. nmap and jq placed through a package manager: apt install nmap jq
Editor: nano for beginners, vim for those who spent an hour studying :wq. Each script begins with a shebang #!/bin/bash and becomes executed after chmod +x script.sh. Without chmod +x get Permission denied The first trap that everyone goes through
Create a working directory immediately: mkdir -p ~/ctf/scripts && cd ~/ctf/scripts. All the weekend script files will fall into one place and you won't lose results in the middle of the competition.
Overtaking bash ports : from /dev/tcp to nmap wrapper
Works if: Bash is compiled with support /dev/tcp (Kali, Ubuntu, Parrot — by default yes), the target host is available by TCP. Does not work if: used dash or sh instead of bash ( in Debian /bin/sh refers to dash, where /dev/tcp missing), Bash collected without --enable-net-redirections, Target behind the firewall with DROP policy.
When nmap unavailable — and on CTF infrastructure with tool restrictions, it happens regularly — Bash is able to check TCP ports through a virtual device /dev/tcp. This is not a file on the disk, but a built-in shell mechanism: writing in /dev/tcp/host/port initiates a TCP connection. Port open - return code 0, closed - error.
#!/bin/bash
HOST="$1"
for port in $(seq 1 1024); do
(echo >/dev/tcp/"$HOST"/"$port") 2>/dev/null && \
echo "[+] $HOST:$port open"
done
Script accepts IP first argument ($1). Cycle seq 1 1024 Searches the first thousand ports. Construction echo >/dev/tcp/"$HOST"/"$port" tries to open TCP connection, brackets () create subshell (isolate the error). Redirect 2>/dev/null suppresses reports of closed ports. Operator && performs echo only with a successful connection.
The problem is the script is consistent. At 1024 ports with a default timeout (about a second to a closed port) it will take up to 17 minutes. On the CTF it is eternity. The solution is parallelization. Add & after echo in the body of the cycle, and after done put wait. Ampersand sends each check to the background process, wait Waiting for the completion of all. For 1024 ports, time drops to 3-5 seconds.
But here is the second trap: 1024 simultaneous processes the machine will survive, and 65535 - most likely not. Limit the parallelism: xargs -P 50 -I{} bash -c 'echo >/dev/tcp/"$1"/{} 2>/dev/null && echo "[+] $1:{} open"' _ "$HOST" starts a maximum of 50 checks at a time. Or GNU parallel with the key -j 50.
When /dev/tcp does not save
UDP Ports: /dev/tcp works only with TCP. For UDP you need nmap -sU or nc -u -w 1 $HOST $port
Definition of services: Clean Bash does not send banner requests. You will learn that port 22 is open, but you will not define the OpenSH version. For versions — nmap -sV or manual nc -w 2 $HOST $port < /dev/null
Timeouts on DROP firewalls: if the firewall drops the packets (does not respond to the RST), each closed port hangs until the timeout. There is no built-in timeout for Bash /dev/tcp – can be wrapped through timeout 1 bash -c "echo >/dev/tcp/$HOST/$port", but it slows down execution and produces daughter processes
Detection: any successive port overtake is detected elementary. In SigmaHQ 19 rules for T1059.004 (Unix Shell), including lnx_shell_susp_commands.yml. Port Scan category (#14 according to the AbuseIPDB classification) is a standard trigger for automatic locks. On CTF it is usually not a problem, on a real pentest - consider
Parsing withdrawal commands linux: grep, awk and sed in combat
Works if: GNU grep is installed (Kali, Ubuntu – default), file in text format. Does not work if: used BSD grep without -P (macOS - replace with grep -oE), binary file (use strings before grep).
The main task in the CTF is not to run the scanner, but to quickly pull the desired one out of its output. Russian-speaking guides usually end on grep "open" – it covers the percentage of ten real tasks of parsing. Let's analyze the processing of the output nmap and curl – two tools that generate the main flow of text data at competitions.
Disassembly nmap -oG via grep and awk
The key flag — -oG (grepable output). Format nmap -oG gives one line to the host with all the open ports – ideal for conveyor processing. Standard multi-line conclusion nmap parsing is inconvenient – each port on a separate line, between hosts empty lines and decorative frames.
Launch: nmap -sC -sV -oG scan.grep -oN scan.txt 10.10.0.0/24. Flag -oG saves the greepable format, -oN – normal (for reading with eyes). Two formats at the same time is a habit that saves time on CTF. I always do that.
Extract open port 80 IP addresses: grep '80/open' scan.grep | awk '{print $2}'. Here awk '{print $2}' outputs the second line field — IP address (first field in format -oG – word Host
.Get a list of all open ports of a particular host: grep '10.10.0.5' scan.grep | grep -oP '\d+/open' | cut -d'/' -f1. Regularity \d+/open through grep -oP (Perl-compatible regulars) extracts the port number before /open, eh cut -d'/' -f1 cuts everything off after the slash. On macOS -P not supported – replace with grep -oE '[0-9]+/open'.
Script for mass processing of greepable-output:
#!/bin/bash
nmap -oG, выводит ip
grep '/open/' "$1" | while read -r line; do
ip=$(echo "$line" | awk '{print $2}')
ports=$(echo "$line" | grep -oP '\d+/open' | \
cut -d'/' -f1 | tr '\n' ',')
echo "$ip -> $ports"
done
Accepts the file nmap -oG The first argument. For each line with open ports, IP is extracted via awk and forms a compact list of ports. Team tr '\n' ',' replaces translations of lines with commas - the output of the view is obtained 10.10.0.5 -> 22,80,443,.
A few patterns sed, which on the CTF are constantly useful. Removing blank lines: sed '/^$/d' output.txt. Extraction of value after password=: sed -n 's/.*password[=:]\s*\([^ ]*\).*/\1/p' config.txt. Break the line hash:salt:user on separate lines: echo "hash:salt:user" | sed 's/:/\n/g'. Output lines from 10 to 20 from the log: sed -n '10,20p' access.log.
Extract data from HTTP responses
On CTF, you often need to process dozens of HTTP answers: find a different length, pull the token out of the title, filter out answers with a certain status.
Basic pattern – curl -s -o /dev/null -w "%{http_code} %{size_download}" – returns HTTP code and response size in bytes without body. Turning into a cycle with a wordlist, you get a simple fuzzer directories:
while read -r dir; do code=$(curl -s -o /dev/null -w "%{http_code}" "http://target/$dir"); [ "$code" != "404" ] && echo "$code $dir"; done < wordlist.txt
Square Brackets in [ "$code" != "404" ] – call the team test. Gaps around brackets and operator are mandatory. Recording ["$code" without space, and hello, [: command not found. The most annoying syntactic feature of Bash and the most frequent question from beginners on forums.
For JSON replies — jq: curl -s http://target/api/user | jq -r '.token'. Without jq can be through grep -oP '"token":"[^"]+"' | cut -d'"' -f4, but it’s fragile – regularly breaks down on nested quotes. For the headlines: curl -sI http://target | grep -i 'set-cookie' – flag -I requests only headings (HEAD request).
Mass processing of bash files: hunting for flags
Works if: has read-access to target directories, GNU grep supports -r. Does not work if: file system encrypted, files in non-standard encoding (UTF-16 — grep will not see ASCII strings without prior conversion through iconv).
In MITRE ATT&CK it is the intersection of two techniques: File and Directory Discovery (T1083, Discovery Tactics) – transfer of files through find and ls – and Automated Collection (T1119, Collection tactics) – automated data collection from found files. In CTF context: you got shell and are looking for a flag, or downloaded the file system dump and processing it locally.
Recursive search by file system
Classic single-line from CTF-chit-shits (HackTricks, PayloadsAllTheThings): grep -rE 'flag\{|CTF\{|HTB\{|THM\{' / 2>/dev/null. Flag -r – recursive search, -E – extended regulars, 2>/dev/null – suppression of mistakes Permission denied. Problem: On a live system with thousands of files, it's slow and generates noise in the logs.
The optimized option is to limit the search area to typical CTF directories:
#!/bin/bash
DIRS="/root /home /opt /var/www /tmp /srv"
PATTERN='flag\{|CTF\{|HTB\{|THM\{|picoCTF\{'
for d in $DIRS; do
[ -d "$d" ] && grep -rlE "$PATTERN" "$d" 2>/dev/null
done
Checking [ -d "$d" ] excludes non-existent ways (without it grep will work, but will issue an error). Flag -l displays only file names with matches, without the lines themselves - faster and more compact. Variable DIRS – typical places where CTF platforms hide flags: home directories of users, /opt (Custom applications), /var/www (Web root) /tmp (temporary files).
For binary files grep Default outputs Binary file matches. Add the flag -a (process as text) or better: strings "$file" | grep -E "$PATTERN" – utility strings extracts printed sequences from binary, and further grep works with clean text. Team find / -type f -exec file {} + | grep 'ELF' find all the ELF binari, and strings with grep will handle each.
Batch processing of dumps and archives
On forensic tasks, a directory with hundreds of files of an unknown type is often found: dumps, archives, images. The task is to process everyone automatically.
Pattern with find and -exec: find ./dump -type f -name '*.gz' -exec gunzip {} \; Unpacks everything .gz-files. But on the CTF extension regularly lie – file image.png may be a ZIP archive. Trust magic bytes, not extension:
find ./dump -type f | while read -r f; do file "$f" | grep -q 'gzip' && gunzip "$f"; done
Team file "$f" determine the type by magic bytes (the first bytes of the file), grep -q 'gzip' checks the result, and only then the unpacking is started.
Critical trap: file names with spaces. Construction while read -r f breaks the default input by spaces and line translations. File my flag.txt it will become two arguments: my and flag.txt. Solution: find ... -print0 | while IFS= read -r -d '' f – zero byte separator instead of translating the line. This is one of the most insidious mistakes – the script silently passes files with spaces, and you lose the flag without knowing it. I lost half an hour on this one.
To determine the distribution of file types in the directory: find ./dump -type f -exec file {} + | awk -F: '{print $2}' | sort | uniq -c | sort -rn. Shows how much PNG, ELF, text, gzip — and helps to understand what to grab first.
Bash one-liner for hacking: working single-liners
Single-liners are a working tool of CTF-cate. Each solves a specific problem and is placed in one line of the terminal. Copied, set the goal, launched.
Search for SUID binarys to escalate privileges: find / -perm -4000 -type f 2>/dev/null. Flag -perm -4000 searching for files with SUID-bit installed. The result is checked by GTFOBins (gtfobins.github.io) — Unix-binary catalog for abuse in post-exploitation. From frequent finds: awk, bash, base64, cat – everyone can read files or give shell if there is a SUID.
System intelligence one team (T1082, System Information Discovery): echo "=== OS ==="; uname -a; echo "=== Users ==="; cat /etc/passwd | grep -v nologin; echo "=== Net ==="; ip a; echo "=== Procs ==="; ps aux --forest. Six commands in one line, instead of six separate runs.
Overtaking subdomains: while read -r sub; do host "$sub.target.com" | grep -v 'NXDOMAIN' && echo "[+] $sub"; done < subdomains.txt. Team host Resolvit DNS, grep -v 'NXDOMAIN' filters non-existent.
Generation of numerical wordlist: for i in $(seq -w 0000 9999); do echo $i; done > pins.txt. Flag -w in seq adds the leading zeros – without it 0001 generated as 1, and the four-digit PIN does not match the format.
Page Change Monitoring: watch -n 5 'curl -s http://target/score | md5sum'. Shows MD5 hash pages every 5 seconds – the hash has changed, so the contents have been updated.
Decoding Base64: cat encoded.txt | base64 -d. For URL-safe Base64 (where + replaced by -, eh / on _: cat encoded.txt | tr '_-' '/+' | base64 -d.
Writing scripts for network scanning: wrapper-pattern
Automation of CTF in practice is not a monolithic script for 200 lines, but a set of wrappers. Each turns a specific tool and transmits the result to the following. Pattern: input data — from arguments or file, the result — to the file for the next script.
#!/bin/bash
set -euo pipefail
TARGET="$1"
OUT="./results/$TARGET"
mkdir -p "$OUT"
echo "[$(date +%H:%M:%S)] Scanning $TARGET..."
nmap -sC -sV -oG "$OUT/nmap.grep" -oN "$OUT/nmap.txt" "$TARGET"
echo "[$(date +%H:%M:%S)] Extracting ports..."
grep '/open/' "$OUT/nmap.grep" | grep -oP '\d+/open' | \
cut -d'/' -f1 > "$OUT/ports.txt"
echo "[*] Found $(wc -l < "$OUT/ports.txt") open ports"
The script creates a results directory for each host, triggers nmap with two output formats, extracts open ports into a separate file. Team wc -l < "$OUT/ports.txt" counts the lines — redirection through < instead of cat file | wc -l Avoids the output of the file name as a result.
The line set -euo pipefail – three protective flags, and everyone saved me scripts more than once. set -e – interruption at the first error (without it the script will continue after the fallen nmap and the next step will get an empty file). set -u – error when referring to an uninitialized variable (typo $TAGET instead of $TARGET silently sets up an empty line, and nmap scan localhost – a pleasant surprise). set -o pipefail – the conveyor returns the error code of the last fallen command, not the last one in the chain.
Further plug in modular scripts: web_enum.sh Launches gobuster dir the HTTP Ports from ports.txt, grab_banners.sh collects banners through nc. Everyone reads a file from the results directory and writes there. Verification of arguments at the beginning of each script: [ -z "$1" ] && echo "Usage: $0 <target>" && exit 1 – without this, the script will start with an empty variable and give an incomprehensible error on the third line.
Temporary labels through echo "[$(date +%H:%M:%S)]" - not the decoration. On a CTF with limited time, they help to understand what step the script is stuck in if the process is hovering. The difference between “script works” and “script hovered on nmap for 8 minutes” is one line with date.
When bash scripts for pentest are not enough
Bash is a glue between utilities, not a full-fledged programming language. And knowing its boundaries saves hours.
Multithreading with control. Background processes through & – rough mechanism. There is no pool of flows, no task queue. For the selection of 65535 ports with a limit of 50 simultaneous connections - xargs -P 50 or GNU parallel, but logic quickly turns into porridge. In Python concurrent.futures.ThreadPoolExecutor solves this in three lines with full control.
Work with JSON/XML. jq covers a simple JSON, but parsing nested structures through chains grep | awk | sed – the path to the fragile code that breaks down every time the format changes. REST API of modern CTF platforms return JSON - you can process them on Bash, but it hurts. Python with json and requests - more reliable.
Complex logic with the state. HTTP session (cookies, CSRF tokens), repeats in errors, redirect chains - the number of Bash strings is growing exponentially. requests.Session() in Python makes it more elegant.
Cryptography. Crypto-tasks (XOR, AES, RSA) through openssl and xxd in Bash – it is possible, but impractical for something more difficult Base64. Python with pycryptodome – standard for crypto on CTF.
Rule for decision: The script has outgrown 30 lines and contains more than three nested if/while – rewrite in Python. Bash is good for a 5-15 line pipeline, where each line is a utility call through a pipeline. It’s more complicated – not its territory.
In terms of detection: Protective systems track suspicious shell activity. In SigmaHQ 19 of the Detection Rules for T1059.004 (Unix Shell), including lnx_shell_susp_rev_shells.yml to detect the reverse shell and lnx_shell_susp_commands.yml for suspicious teams. Countermeasures on D3FEND include Executable Denyllisting (D3-EDL) – whitelist execution blocking – and Content Filtering (D3-CF). On the CTF, these mechanisms are usually switched off, on the real Bash script pentest can be blocked by EDR before the first execution.
At each competition I see the same picture: teams are divided into those who write scripts for the task in minutes, and those who spend these minutes on manual input. The difference in the final score is not in knowing the vulnerabilities, but in the speed of information processing. Bash covers 80% of the routine on the CTF: ports, parsing output, search for flags on the file system. The remaining 20% — complex logic, crypto, work with API — go Python. But the skill to assemble the conveyor from grep | awk | sort | uniq in 30 seconds, while the opponent reads the manual - does not appear from reading articles. It appears after the 50-th task when you start to think of conveyors. After 200, you see the patterns in the output before you start grep. If you go to OSCP and need intelligence and scripting training, WAPT covers this in the first modules with labs for each case.
