Advanced Shell Concepts¶
Globbing (Wildcards)¶
Globbing interprets patterns to match files and directories. The wildcards available are *, ?, and [ ].
* - Match Any Characters¶
Matches zero or more characters:
ls *.py # All Python files
ls data_* # All files starting with "data_"
ls *.txt.bak # All .txt.bak files
? - Match Single Character¶
Matches exactly one character:
ls file?.txt # Matches file1.txt, fileA.txt, file8.txt
ls report?.pdf # Matches report1.pdf through report9.pdf
Does NOT match file12.txt (two characters where ? is).
[ ] - Match Character Set¶
Matches exactly one character from the set:
ls image[12].png # Matches image1.png or image2.png
ls photo[abc].jpg # Matches photoa.jpg, photob.jpg, or photoc.jpg
Ranges¶
Use - to specify ranges:
ls image[1-10].png # Matches image1.png through image10.png
ls report[a-z].txt # Matches reporta.txt through reportz.txt
ls file[0-9].dat # Matches file0.dat through file9.dat
Negation¶
Use ! or ^ to match characters NOT in the set:
ls *.![ch] # All files except .c and .h files
ls file[^0-9]* # Files starting with "file" but NOT followed by a digit
Pipes and Redirection¶
Standard Streams¶
Every command has three default data streams:
Stream |
Abbreviation |
Description |
Default |
|---|---|---|---|
Standard Input |
|
Where input comes from |
Keyboard |
Standard Output |
|
Where results go |
Terminal |
Standard Error |
|
Where errors go |
Terminal |
Pipes (|)¶
Pipes connect commands by feeding the stdout of one command as stdin to the next:
ls | grep file.txt # List files, then filter for "file.txt"
ps aux | grep python # Show all processes, filter for Python
cat log.txt | head -20 # Show first 20 lines of log
Data only flows left to right through pipes.
Output Redirection (> and >>)¶
>- Redirect output to file (overwrites if exists)>>- Append output to file (creates if doesn’t exist)
echo "Hello" > hello.txt # Create/overwrite hello.txt
date >> datelog.txt # Append date to datelog.txt
ls -l > listing.txt # Save directory listing
Warning
command > file will overwrite file without warning.
Input Redirection (<)¶
Feed a file as input to a command:
sort < names.txt # Sort contents of names.txt
wc < input.txt # Count lines/words in input.txt
Error Redirection (2>)¶
Redirect error messages separately:
./script.sh 2> errors.log # Save errors to errors.log
command > output.txt 2>&1 # Redirect both stdout and stderr to output.txt
command > /dev/null 2>&1 # Discard all output
Combining Pipes and Redirection¶
ls -la | grep "\.py$" > python_files.txt
ps aux | grep nginx | awk '{print $2}' > nginx_pids.txt
cat *.log | grep ERROR | tail -50 > recent_errors.txt
Common Patterns¶
Filter and Process¶
# Find all Python files and count them
ls *.py | wc -l
# Show only lines containing "error" from a log
grep -i error application.log
# Sort a file and remove duplicates
sort file.txt | uniq > sorted_unique.txt #or sort -u
Save Intermediate Results¶
# Chain multiple operations
ls -la | grep "^d" > directories.txt # Save only directories
ls -la | grep "^-" > files.txt # Save only regular files
Redirect Both Output and Errors¶
# Send everything to one file
command > all_output.txt 2>&1
# Modern equivalent
command &> all_output.txt
# Discard everything
command > /dev/null 2>&1