Shell Scripting Recipes
A Problem-Solution Approach

Chris F.A. Johnson

Purchase from:
Amazon.com
Amazon.ca
Amazon.co.uk
E-book
Contents
About the Authorxv
About the Technical Reviewerxvii
Acknowledgmentsxix
Introductionxxi

CHAPTER 1:
The POSIX Shell and Command-Line Utilities

Shell Commands1
Parameters and Variables7
    Positional Parameters7
 Special Parameters7
standard-vars — A Collection of Useful Variables9
Patterns9
 Filename expansion10
 Regular Expressions10
Parameter Expansion12
 The Bourne Shell Expansions12
 POSIX Parameter Expansions14
 Shell-Specific Expansions, bash2, and ksh9315
Shell Arithmetic16
Aliases16
Sourcing a File17
Functions17
 Functions Are Fast17
 Command Substitution Is Slow17
 Using the Functions in This Book18
standard-funcs: A Collection of Useful Commands19
 1.1 get_key — Get a Single Keystroke from the User19
 1.2 getline — Prompt User to Enter a Line20
 1.3 press_any_key — Prompt for a Single Keypress21
 1.4 menu1 — Print a Menu and Execute a Selected Command22
 1.5 arg — Prompt for Required Argument If None Supplied23
 1.6 die — Print Error Message and Exit with Error Status24
 1.7 show_date — Display Date in D[D] MMM YYYY Format25
 1.8 date_vars — Set Date and Time Variables26
 1.9 is_num — Is This a Positive Integer?28
 1.10 abbrev_num — Abbreviate Large Numbers29
 1.11 commas — Add Thousands Separators to a Number30
 1.12 pr1 — Print Arguments, One to a Line32
 1.13 checkdirs — Check for Directories; Create If Necessary33
 1.14 checkfiles — Check That a Directory Contains Certain Files33
 1.15 zpad — Pad a Number with Leading Zeroes34
 1.16 cleanup — Remove Temporary Files and Reset Terminal on Exit35
The Unix Utilities36
Summary40

CHAPTER 2
Playing with Files: Viewing, Manipulating, and Editing Text Files

End of the Line for OS Differences: Converting Text Files42
 2.1 dos2unix — Convert Windows Text Files to Unix42
 2.2 unix2dos — Convert a Unix File to Windows43
 2.3 mac2unix — Convert Macintosh Files to Unix44
 2.4 unix2mac — Convert Unix Files to Mac Format44
 2.5 dos2mac — Convert Windows Files to Macintosh45
 2.6 mac2dos — Convert Macintosh Files to Windows45
Displaying Files46
 2.7 prn — Print File with Line Numbers46
 2.8 prw — Print One Word per Line49
 2.9 wbl — Sort Words by Length51
Formatting File Facts52
 2.10 finfo — File Information Display52
 2.11 wfreq — Word Frequency56
 2.12 lfreq — Letter Frequency56
 2.13 fed — A Simple Batch File Editor58
Summary60

CHAPTER 3
String Briefs

Character Actions: The char-funcs Library61
 3.1 chr — Convert a Decimal Number to an ASCII Character62
 3.2 asc — Convert a Character to Its Decimal Equivalent63
 3.3 nxt — Get the Next Character in ASCII Sequence64
 3.4 upr — Convert Character(s) to Uppercase65
 3.5 lwr — Convert Character(s) to Lowercase67
String Cleaning: The string-funcs Library69
 3.6 sub — Replace First Occurrence of a Pattern69
 3.7 gsub — Globally Replace a Pattern in a String70
 3.8 repeat — Build a String of a Specified Length72
 3.9 index, rindex — Find Position of One String Within Another74
 3.10 substr — Extract a Portion of a String76
 3.11 insert_str — Place One String Inside Another78
Summary79

CHAPTER 4
What's in a Word?

Finding and Massaging Word Lists82
wf-funcs: WordFinder Function Library82
 4.1 write_config — Write User's Information to the Configuration File82
 4.2 do_config — Check For and Source Default Configuration File83
 4.3 set_sysdict — Select the Dictionary Directory84
 4.4 mkwsig — Sort Letters in a Word85
 4.5 wf-clean — Remove Carriage Returns and Accents86
 4.6 wf-compounds — Squish Compound Words and Save with Lengths87
 4.7 wf-setup — Prepare Word and Anagram Lists89
Playing with Matches91
 4.8 wf — Find Words That Match a Pattern91
 4.9 wfb — Find Words That Begin with a Given Pattern93
 4.10 wfe — Find Words That End with a Given Pattern94
 4.11 wfc — Find Words That Contain a Given Pattern95
 4.12 wfit — Find Words That Fit Together in a Grid97
 4.13 anagram — Find Words That Are Anagrams of a Given Word100
 4.14 aplus — Find Anagrams of a Word with a Letter Added101
 4.15 aminus — Remove Each Letter in Turn and Anagram What's Left103
Summary104

CHAPTER 5
Scripting by Numbers

The math-funcs Library105
 5.1 fpmul — Multiply Decimal Fractions106
 5.2 int — Return the Integer Portion of a Decimal Fraction110
 5.3 round — Round the Argument to the Nearest Integer110
 5.4 pow — Raise a Number to Any Given Power111
 5.5 square — Raise a Number to the Second Power113
 5.6 cube — Raise a Number to the Third Power113
 5.7 calc — A Simple Command-Line Calculator114
Adding and Averaging115
 5.8 total — Add a List of Numbers115
 5.9 mean — Find the Arithmetic Mean of a List of Numbers116
 5.10 median — Find the Median of a List of Numbers118
 5.11 mode — Find the Number That Appears Most in a List119
 5.12 range — Find the Range of a Set of Numbers120
 5.13 stdev — Finding the Standard Deviation122
Converting Between Unit Systems123
 5.14 conversion-funcs — Converting Metric Units124
 5.15 conversion — A Menu System for Metric Conversion131
Summary135

CHAPTER 6
Loose Names Sink Scripts: Bringing Sanity to Filenames

What's in a Name?137
 POSIX Portable Filenames138
 OK Filenames138
Functioning Filenames: The filename-funcs Library139
 6.1 basename — Extract the Last Element of a Pathname139
 6.2 dirname — Return All but the Last Element of a Pathname141
 6.3 is_pfname — Check for POSIX Portable Filename142
 6.4 is_OKfname — Check Whether a Filename Is Acceptable143
 6.5 pfname — Convert Nonportable Characters in Filename144
 6.6 OKfname — Make a Filename Acceptable145
 6.7 is_whitespc — Does the Filename Contain Whitespace Characters?146
 6.8 whitespc — Fix Filenames Containing Whitespace Characters146
 6.9 is_dir — Is This a Directory I See Before Me?147
 6.10 nodoublechar — Remove Duplicate Characters from a String148
 6.11 new_filename — Change Filename to Desired Character Set149
 6.12 fix_pwd — Fix All the Filenames in the Current Directory150
 6.13 fixfname — Convert Filenames to Sensible Names152
Summary154

CHAPTER 7
Treading a Righteous PATH

The path-funcs Library157
 7.1 path — Display a User's PATH157
 7.2 unslash — Remove Extraneous Slashes158
 7.3 checkpath — Clean Up the PATH Variable159
 7.4 addpath — Add Directories to the PATH Variable161
 7.5 rmpath — Remove One or More Directories from $PATH163
Summary164

CHAPTER 8
The Dating Game (Sample chapter: PDF or HTML)

The date-funcs Library166
 8.1 split_date — Divide a Date into Day, Month, and Year166
 8.2 is_leap_year — Is There an Extra Day This Year?167
 8.3 days_in_month — How Many Days Hath September?169
 8.4 date2julian — Calculate the Julian Day Number170
 8.5 julian2date — Convert Julian Back to Year, Month, and Day170
 8.6 dateshift — Add or Subtract a Number of Days172
 8.7 diffdate — Find the Number of Days Between Two Dates174
 8.8 day_of_week — Find the Day of the Week for Any Date175
 8.9 display_date — Show a Date in Text Format176
 8.10 parse_date — Decipher Various Forms of Date String179
 8.11 valid_date — Where Was I on November 31st?183
Summary184

CHAPTER 9
Good Housekeeping: Monitoring and Tidying Up File Systems

 9.1 dfcmp — Notify User of Major Changes in Disk Usage185
 9.2 symfix — Remove Broken Symbolic Links189
 9.3 sym2file — Converts Symbolic Links to Regular Files191
 9.4 zrm — Remove Empty Files193
 9.5 undup — Remove Duplicate Files194
 9.6 lsr — List the Most Recent (or Oldest) Files in a Directory196
Summary198

CHAPTER 10
POP Goes the E-Mail

 10.1 popcheck — Display Number of Messages in POP3 Mailbox199
 10.2 pop3-funcs — Functions for Managing a POP3 Mailbox200
 10.3 pop3list — Examine a POP3 Mailbox215
 10.4 pop3filter — Delete and Retrieve Messages from a POP3 Server222
 10.5 pflog — Print a Summary of a Day's Activity224
 10.6 viewlog — Continuously View the pop3filter Log File225
Summary227

CHAPTER 11
PostScript: More Than an Afterthought

 11.1 ps-labels — Produce Mailing Labels from a List231
 11.2 ps-envelopes — Print Envelopes from List237
 11.3 ps-grid — Convert a Crossword Grid to PostScript242
Summary249

CHAPTER 12
Screenplay: The screen-funcs Library

 12.1 screen-vars — Variables for Screen Manipulation252
 12.2 set_attr — Set Screen-Printing Attributes254
 12.3 set_fg, set_bg, set_fgbg — Set Colors for Printing to the Screen255
 12.4 cls — Clear the Screen256
 12.5 printat — Position Cursor by Row and Column257
 12.6 put_block_at — Print Lines in a Column Anywhere on the Screen260
 12.7 get_size — Set LINES and COLUMNS Variables262
 12.8 max_length — Find the Length of the Longest Argument263
 12.9 print_block_at — Print a Block of Lines Anywhere on the Screen264
 12.10 vbar, hbar — Print a Vertical or Horizontal Bar265
 12.11 center — Center a String on N Columns267
 12.12 flush_right — Align String with the Right Margin268
 12.13 ruler — Draw a Ruler Across the Width and Height of the Window269
 12.14 box_block, box_block_at — Print Text Surrounded by a Box271
 12.15 clear_area, clear_area_at — Clear an Area of the Screen272
 12.16 box_area, box_area_at — Draw a Box Around an Area273
 12.17 screen-demo — Saving and Redisplaying Areas of the Screen275
Summary277

CHAPTER 13
Backing Up the Drive

 13.1 bu — Backup Directories280
 13.2 bin-pack — Fill Fixed-Capacity Directories with Files285
 13.3 unpack — Extract All Files from One or More Archives290
Summary292

CHAPTER 14
Aging, Archiving, and Deleting Files

 14.1 date-file — Add a Datestamp to a Filename293
 14.2 rmold — Remove Old Files297
 14.3 keepnewest — Remove All but the Newest or Oldest Files300
Summary302

CHAPTER 15
Covering All Your Databases

 15.1 lookup — Find the Corresponding Value for a Key304
 shdb-funcs: Shell Database Function Library306
 15.2 load_db — Import Database into Shell Array306
 15.3 split_record — Split a Record into Fields307
 15.4 csv_split — Extract Fields from CSV Records309
 15.5 put_record — Assemble a Database Record from an Array310
 15.6 put_csv — Assemble Fields into a CSV Record311
 15.7 db-demo — View and Edit a Password File312
PhoneBase: A Simple Phone Number Database317
 15.8 ph — Look Up a Phone Number318
 15.9 phadd — Add an Entry to PhoneBase319
 15.10 phdel — Delete an Entry from PhoneBase320
 15.11 phx — Show ph Search Results in an X Window320
Summary321

CHAPTER 16
Home on the Web

Playing with Hypertext: The html-funcs Library323
 16.1 get_element — Extract the First Occurrence of an Element323
 16.2 split_tags — Put Each Tag on Its Own Line326
 16.3 html-title — Get the Title from an HTML File327
HTML on the Fly: The cgi-funcs Library329
 16.4 x2d2 — Convert a Two-Digit Hexadecimal Number to Decimal329
 16.5 dehex — Convert Hex Strings (%XX) to Characters330
 16.6 filedate — Find and Format the Modification Date of a File332
Creating HTML Files333
 16.7 mk-htmlindex — Create an HTML Index333
 16.8 pretext — Create a Wrapper Around a Text File341
 16.9 text2html — Convert a Text File to HTML342
 16.10 demo.cgi — A CGI Script343
Summary344

CHAPTER 17
Taking Care of Business

 17.1 prcalc — A Printing Calculator345
 17.2 gle — Keeping Records Without a Shoebox350
Summary360

CHAPTER 18
Random Acts of Scripting

The rand-funcs Library361
 18.1 random — Return One or More Random Integers in a Given Range362
 18.2 toss — Simulate Tossing a Coin365
 18.3 randstr — Select a String at Random366
A Random Sampling of Scripts367
 18.4 rand-date — Generate Random Dates in ISO Format367
 18.5 randsort — Print Lines in Random Order369
 18.6 randomword — Generate Random Words According to Format Specifications370
 18.7 dice — Roll a Set of Dice375
 18.8 throw — Throw a Pair of Dice378
Summary379

CHAPTER 19
A Smorgasbord of Scripts

 19.1 topntail — Remove Top and Bottom Lines from a File381
 19.2 flocate — Locate Files by Filename Alone383
 19.3 sus — Display a POSIX Man Page384
 19.4 cwbw — Count Words Beginning With385
 19.5 cci — Configure, Compile, and Install from Tarball386
 19.6 ipaddr — Find a Computer's Network Address387
 19.7 ipaddr.cgi — Print the Remote Address of an HTTP Connection388
 19.8 iprev — Reverse the Order of Digits in an IP Address389
 19.9 intersperse — Insert a String Between Doubled Characters389
 19.10 ll — Use a Pager for a Directory Listing Only If Necessary390
 19.11 name-split — Divide a Person's Full Name into First, Last, and Middle Names391
 19.12 rot13 — Encode or Decode Text392
 19.13 showfstab — Show Information from /etc/fstab393
 19.14 unique — Remove All Duplicate Lines from a File395
Summary395

CHAPTER 20
Script Development Management

 20.1 script-setup — Prepare the Scripting Environment397
 20.2 cpsh — Install Script and Make Backup Copy402
 20.3 shgrep — Search Scripts for String or Regular Expression403
 20.4 shcat — Display a Shell Script404
Summary405

APPENDIX
Internet Scripting Resources

Introductions to Shell Scripting407
Intermediate and Advanced Scripting408
Collections of Scripts408
Home Pages for Shells408
Regular Expressions, sed, and awk408
Miscellaneous Pages409
History of the Shell409

INDEX