Wednesday, July 8, 2009

Sed Tips

sed (Stream EDitor) refers to a Unix utility for parsing text files and the programming language it uses to apply textual transformations to a sequential stream of data. It reads input files line by line, applying the operation which has been specified via the command line (or a sed script), and then outputs the line. Getting started with sed can be a real pain if you are unfamiliar with perl for regular expressions.

FILE SPACING:

Double space a file
sed '/^$/d;G'

Undo double-spacing (assumes even-numbered lines are always blank)
sed 'n;d'

NUMBERING:

Number each line of a file (number on left, right-aligned)
sed = filename | sed 'N; s/^/ /; s/ *(.{6,})n/1 /'

Number each line of file, but only print numbers if line is not blank
sed '/./=' filename | sed '/./N; s/n/ /'

Count lines (emulates “wc -l”)
sed -n '$='

TEXT CONVERSION AND SUBSTITUTION:

Convert DOS newlines (CR/LF) to Unix format.
sed 's/.$//'

Convert Unix newlines (LF) to DOS format.
sed 's/$'"/`echo r`/"

Delete leading whitespace (spaces, tabs) from front of each line aligns all text flush left
sed 's/^[ t]*//’

Delete trailing whitespace (spaces, tabs) from end of each line
sed 's/[ t]*$//’

Delete BOTH leading and trailing whitespace from each line
sed 's/^[ t]*//;s/[ t]*$//’

Substitute (find and replace) “foo” with “bar” on each line
sed 's/foo/bar/' # replaces only 1st instance in a line

sed 's/foo/bar/4' # replaces only 4th instance in a line

sed 's/foo/bar/g' # replaces ALL instances in a line

Change “scarlet” or “ruby” or “puce” to “red”
sed 's/scarlet/red/g;s/ruby/red/g;s/puce/red/g'

SELECTIVE PRINTING OF CERTAIN LINES:

Print first 10 lines of file (emulates behavior of “head”)
sed 10q

Print first line of file (emulates “head -1″)
sed q

Print the last 10 lines of a file (emulates “tail”)
sed -e :a -e '$q;N;11,$D;ba'

Print the last 2 lines of a file (emulates “tail -2″)
sed '$!N;$!D'

Print the last line of a file (emulates “tail -1″)
sed -n '$p'

Print section of file based on line numbers (lines 8-12, inclusive)
sed -n '8,12p'

Print only lines which match regular expression (emulates “grep”)
sed '/regexp/!d'

Grep for AAA and BBB and CCC (in any order)
sed '/AAA/!d; /BBB/!d; /CCC/!d'

Grep for AAA and BBB and CCC (in that order)
sed '/AAA.*BBB.*CCC/!d'

SELECTIVE DELETION OF CERTAIN LINES:

Delete duplicate, consecutive lines from a file (emulates “uniq”).
sed '$!N; /^(.*)n1$/!P; D'

Delete duplicate, nonconsecutive lines from a file.
sed -n 'G; s/n/&&/; /^([ -~]*n).*n1/d; s/n//; h; P’

Delete all lines except duplicate lines (emulates “uniq -d”).
sed '$!N; s/^(.*)n1$/1/; t; D'

Delete the first line of a file
sed '1d'

Delete the first 10 lines of a file
sed '1,10d'

Delete the last line of a file
sed '$d'

Delete the last 2 lines of a file
sed 'N;$!P;$!D;$d'

Delete the last 10 lines of a file
sed -n -e :a -e '1,10!{P;N;D;};N;ba'

Delete lines matching pattern
sed '/pattern/d'

Delete ALL blank lines from a file (same as “grep ‘.’ “)
sed '/^$/d'

Remove most HTML tags (accommodates multiple-line tags)
sed -e :a -e 's/<[^>]*>//g;/

No comments:

Post a Comment