sed: -i may not be used with stdin on Mac OS X
You need to put the input file as the last parameter.
sed -i -e "s/ __attribute__ ((__unused__))$/# ifndef __cplusplus\n __attribute__ ((__unused__));\n# endif/" y.tab.c
sed: -i may not be used with stdin #Error
Asuming that you're trying to sed something only in the files that contain xyz, you will have to xargs again
find . -name "*" -type f | xargs grep -l "xyz" |xargs sed -i "s/'${line}'/'${rep}'/g"
In-place edits with sed on OS X
You can use the -i
flag correctly by providing it with a suffix to add to the backed-up file. Extending your example:
sed -i.bu 's/oldword/newword/' file1.txt
Will give you two files: one with the name file1.txt
that contains the substitution, and one with the name file1.txt.bu
that has the original content.
Mildly dangerous
If you want to destructively overwrite the original file, use something like:
sed -i '' 's/oldword/newword/' file1.txt
^ note the space
Because of the way the line gets parsed, a space is required between the option flag and its argument because the argument is zero-length.
Other than possibly trashing your original, I’m not aware of any further dangers of tricking sed this way. It should be noted, however, that if this invocation of sed
is part of a script, The Unix Way™ would (IMHO) be to use sed
non-destructively, test that it exited cleanly, and only then remove the extraneous file.
sed not giving me correct substitute operation for newline with Mac - differences between GNU sed and BSD / OSX sed
With BSD/macOS sed
, to use a newline in the replacement string of an s
function call, you must use an \
-escaped actual newline - escape sequence \n
is not supported there (unlike in the regex part of the call).
Either: simply insert an actual newline:
sed -i '' 's/\\n/\
/g' test1.txtOr: use an ANSI C-quoted string (
$'...'
) to splice in the newline ($'\n'
; works inbash
,ksh
, orzsh
):sed -i '' 's/\\n/\'$'\n''/g' test1.txt
GNU sed
, by contrast, does recognize \n
in replacement strings; read on for a comprehensive overview of the differences between these two implementations.
Differences between GNU sed
(Linux) and BSD/macOS sed
macOS uses the BSD version of sed
[1], which differs in many respects from the GNU sed
version that comes with Linux distros.
Their common denominator is the functionality decreed by POSIX: see the POSIX sed
spec.
The most portable approach is to use POSIX features only, which, however, limits functionality:
- Notably, POSIX specifies support only for basic regular expressions, which have many limitations (e.g., no support for
|
(alternation) at all, no direct support for+
and?
) and different escaping requirements.- Caveat: GNU
sed
(without-r
), does support\|
,\+
and\?
, which is NOT POSIX-compliant; use--posix
to disable (see below).
- Caveat: GNU
- To use POSIX features only:
- (both versions): use only the
-n
and-e
options (notably, do not use-E
or-r
to turn on support for extended regular expressions) - GNU
sed
: add option--posix
to ensure POSIX-only functionality (you don't strictly need this, but without it you could end up inadvertently using non-POSIX features without noticing; caveat:--posix
itself is not POSIX-compliant) - Using POSIX-only features means stricter formatting requirements (forgoing many conveniences available in GNU
sed
):- Control-character sequences such as
\n
and\t
are generally NOT supported. - Labels and branching commands (e.g.,
b
) must be followed by an actual newline or continuation via a separate-e
option. - See below for details.
- Control-character sequences such as
- (both versions): use only the
However, both versions implement extensions to the POSIX standard:
- what extensions they implement differs (GNU
sed
implements more). - even those extensions they both implement partially differ in syntax.
If you need to support BOTH platforms (discussion of differences):
- Incompatible features:
- Use of the
-i
option without an argument (in-place updating without backup) is incompatible:- BSD
sed
: MUST use-i ''
- GNU
sed
: MUST use just-i
(equivalent:-i''
) - using-i ''
does NOT work.
- BSD
-i
sensibly turns on per-input-file line numbering in GNUsed
and recent versions of BSDsed
(e.g., on FreeBSD 10), but does NOT on macOS as of 10.15.
Note that in the absence of-i
all versions number lines cumulatively across input files.- If the last input line does not have a trailing newline (and is printed):
- BSD
sed
: always appends a newline on output, even if the input line doesn't end in one. - GNU
sed
: preserves the trailing-newline status, i.e., it appends a newline only if the input line ended in one.
- BSD
- Use of the
- Common features:
- If you restrict your
sed
scripts to what BSDsed
supports, they will generally work in GNUsed
too - with the notable exception of using platform-specific extended regex features with-E
. Obviously, you'll also forgo extensions that are specific to the GNU version. See next section.
- If you restrict your
Guidelines for cross-platform support (macOS/BSD, Linux), driven by the stricter requirements of the BSD version:
Note that I'm using the shorthands macOS and Linux for the BSD and GNU versions of sed
respectively because they are the stock versions on each platform. However, it is possible to install GNU sed
on macOS, for instance, using Homebrew with brew install gnu-sed
.
Note: Except for when the -r
and -E
flags are used (extended regexes), the instructions below amount to writing POSIX-compliant sed
scripts.
- For POSIX compliance, you must restrict yourself to POSIX BREs (basic regular expressions), which are, unfortunately, as the name suggests, quite basic.
Caveat: do not assume that\|
,\+
and\?
are supported: While GNUsed
supports them (unless--posix
is used), BSDsed
does not - these features are not POSIX-compliant.
While\+
and\?
can be emulated in POSIX-compliant fashion :\{1,\}
for\+
,\{0,1\}
for\?
,\|
(alternation) cannot, unfortunately. For more powerful regular expressions, use
-E
(rather than-r
) to support EREs (extended regular expressions) (GNUsed
doesn't document-E
, but it does work there as an alias of-r
; newer version of BSDsed
, such as on FreeBSD 10, now also support-r
, but the macOS version as of 10.10 does not).
Caveat: Even though use of-r
/-E
means that your command is by definition not POSIX-compliant, you must still restrict yourself to POSIX EREs (extended regular expressions). Sadly, this means that you won't be able to use several useful constructs, notably:- word-boundary assertions, because they're platform-specific (e.g.,
\<
on Linux,[[:<]]
on OS X). - back-references inside regular expressions (as opposed to the "back-references" to capture-group matches in the replacement string of
s
function calls), because BSDsed
doesn't support them in extended regexes (but, curiously, does so in basic ones, where they are POSIX-mandated).
- word-boundary assertions, because they're platform-specific (e.g.,
Control-character escape sequences such as
\n
and\t
:- In regexes (both in patterns for selection and the first argument to the
s
function), assume that only\n
is recognized as an escape sequence (rarely used, since the pattern space is usually a single line (without terminating\n
), but not inside a character class, so that, e.g.,[^\n]
doesn't work; (if your input contains no control chars. other than\t
, you can emulate[^\n]
with[[:print:][:blank:]]
; otherwise, splice control chars. in as literals[2]) - generally, include control characters as literals, either via spliced-in ANSI C-quoted strings (e.g.,$'\t'
) in shells that support it (bash,
ksh,zsh
), or via command substitutions usingprintf
(e.g.,"$(printf '\t')"
).- Linux only:
sed 's/\t/-/' <<<$'a\tb' # -> 'a-b'
- macOS and Linux:
sed 's/'$'\t''/-/' <<<$'a\tb' # ANSI C-quoted string
sed 's/'"$(printf '\t')"'/-/' <<<$'a\tb' # command subst. with printf
- Linux only:
In replacement strings used with the
s
command, assume that NO control-character escape sequences are supported, so, again, include control chars. as literals, as above.- Linux only:
sed 's/-/\t/' <<<$'a-b' # -> 'a<tab>b'
sed 's/-/\n/' <<<$'a-b' # -> 'a<newline>b'
- macOS and Linux:
sed 's/-/'$'\t''/' <<<'a-b'
sed 's/-/'"$(printf '\t')"'/' <<<'a-b'
sed 's/-/\'$'\n''/' <<<'a-b'
Note that newlines need to be backslash-escaped so that they are properly interpreted as part of the replacement string and not as the end of the command, and that usingprintf
does not work for newlines since trailing newlines are removed by command substitutions ($(...)
).
- Linux only:
Ditto for the text arguments to the
i
anda
functions: do not use control-character sequences - see below.
- In regexes (both in patterns for selection and the first argument to the
- Labels and branching: labels as well as the label-name argument to the
b
andt
functions must be followed by either by a literal newline or a spliced-in$'\n'
. Alternatively, use multiple-e
options and terminate each right after the label name.- Linux only:
sed -n '/a/ bLBL; d; :LBL p' <<<$'a\nb' # -> 'a'
- macOS and Linux:
- EITHER (actual newlines):
sed -n '/a/ bLBL
d; :LBL
p' <<<$'a\nb' - OR (spliced-in
$\n
instances):sed -n '/a/ bLBL'$'\n''d; :LBL'$'\n''p' <<<$'a\nb'
- OR (multiple
-e
options):sed -n -e '/a/ bLBL' -e 'd; :LBL' -e 'p' <<<$'a\nb'
- EITHER (actual newlines):
- Linux only:
- Functions
i
anda
for inserting/appending text: follow the function name by\
, followed either by a literal newline or a spliced-in$'\n'
before specifying the text argument.- Linux only:
sed '1 i new first line' <<<$'a\nb' # -> 'new first line<nl>a<nl>b'
- macOS and Linux:
sed -e '1 i\'$'\n''new first line' <<<$'a\nb'
- Note:
- Without
-e
, the text argument is inexplicably not newline-terminated on output in macOS (bug?). - Do not use control-character escapes such as
\n
and\t
in the text argument, as they're only supported on Linux. - If the text argument therefore has actual interior newlines,
\
-escape them. - If you want to place additional commands after the text argument, you must terminate it with an (unescaped) newline (whether literal or spliced in), or continue with a separate
-e
option (this is a general requirement that applies to all versions).
- Without
- Linux only:
Inside function lists (multiple function calls enclosed in
{...}
), be sure to also terminate the last function, before the closing}
, with;
.- Linux only:
sed -n '1 {p;q}' <<<$'a\nb' # -> 'a'
- macOS and Linux:
sed -n '1 {p;q;}' <<<$'a\nb'
With the
-f
option (to read commands from a file), only GNUsed
supports-
as a placeholder for stdin; use-f /dev/stdin
to portably read commands from stdin, including from here-documents (assuming your platform supports/dev/stdin
, which is typically the case nowadays).
GNU sed
-specific features missing from BSD sed
altogether:
GNU features you'll miss out on if you need to support both platforms:
Various regex-matching and substitution options (both in patterns for line selection and the first argument to the
s
function):- The
I
option for case-INsensitive regex matching (incredibly, BSDsed
doesn't support this at all). - The
M
option for multi-line matching (where^
/$
match the start / end of each line) - For additional options that are specific to the
s
function see https://www.gnu.org/software/sed/manual/sed.html#The-_0022s_0022-Command
- The
Escape sequences
Substitution-related escape sequences such as
\u
in the replacement argument of thes///
function that allow substring manipulation, within limits; e.g.,sed 's/^./\u&/' <<<'dog' # -> 'Dog'
- see http://www.gnu.org/software/sed/manual/sed.html#The-_0022s_0022-CommandControl-character escape sequences: in addition to
\n
,\t
, ..., codepoint-based escapes; for instance, all of the following escapes (hex., octal, decimal) represent a single quote ('
):\x27
,\o047
,\d039
- see https://www.gnu.org/software/sed/manual/sed.html#Escapes
Address extensions, such as
first~step
to match every step-th line,addr, +N
to match N lines followingaddr
, ... - see http://www.gnu.org/software/sed/manual/sed.html#Addresses
[1] The macOS sed
version is older than the version on other BSD-like systems such as FreeBSD and PC-BSD. Unfortunately, this means that you cannot assume that features that work in FreeBSD, for instance, will work [the same] on macOS.
[2] The ANSI C-quoted string $'\001\002\003\004\005\006\007\010\011\013\014\015\016\017\020\021\022\023\024\025\026\027\030\031\032\033\034\035\036\037\177'
contains all ASCII control characters except \n
(and NUL), so you can use it in combination with [:print:]
for a pretty robust emulation of [^\n]
:'[[:print:]'$'\001\002\003\004\005\006\007\010\011\013\014\015\016\017\020\021\022\023\024\025\026\027\030\031\032\033\034\035\036\037\177'']
Prevent sed from creating a temporary file
This is because of the order you are placing the flags, if you are under a non-GNU sed that requires -i
to have an argument:
sed -i -e '...'
# ^^ ^^
# | |
# flag|
# |
# value for it
Like this, sed
understands -e
as the extension of the backup file to create.
Instead, say:
sed -e -i '...'
Or just be explicit about the extension and set it to null, so that no backup file is created:
sed -i '' -e '...'
See more information in sed: -i may not be used with stdin on Mac OS X.
Why is sed not able to process piped output from xpath?
I don't have Mac OS but I can guess your problem. If I do the equivalent under Linux I get the following output:
$ curl -s http://wordsmith.org/awad/rss1.xml | xpath -e "//item/description" | sed q
Found 1 nodes in stdin:
-- NODE --
<description>Ending life for humane reasons, such as to avoid pain from an incurable condition.</description>
That's because part of the output is going to stdout and part is going to stderr. So if I redirect everything to stdout, I get this,
$ curl -s http://wordsmith.org/awad/rss1.xml | xpath -e "//item/description" 2>&1 | sed q
Found 1 nodes in stdin:
sed and double quotes
\"*
means *double quote zero or more times. You need a bit different regular expression:
sed -e "s/^\(password = \)\".*\"$/\1\"$password\";/"
Note that case
is suitable if there is more then one possibility, here if
would be more suitable:
while (( $# )) ; do
if [[ $1 == '-password' ]] ; then
password=$2
shift
fi
shift # What does this do, anyway?
done
if grep -q 'password = ' test.cfg ; then
sed -i~ -e "s/^\(password = \)\".*\"$/\1\"$password\";/" test.cfg
else
echo 'password = "'"$password"'";' >> test.cfg
fi
Sed & Mac OS Terminal: How to remove parentheses content from the first line of every file?
I managed to achieve what you're after simply by using a bash script and sed together, as so:
#!/bin/bash
for filename in $PWD/*.txt; do
sed -i '' '1 s/([^()]*)//g' $filename
done
The script simply iterates over all the .txt
files in $PWD (the current working directory, so that you can add this script to your bin and run it anywhere), and then runs the command
sed -ie '1 s/([^()]*)//g' $filename
on the file. By starting the command with the number 1
we tell sed to only work on the first line of the file :)
Edit: Best Answer
The above works fine in a directory where all contained objects are files, and not including directories; in other words, the above does not perform recursive search through directories.
Therefore, after some research, this command should perform exactly what the question asks:
find . -name "*.txt" -exec sed -i '' '1 s/([^()]*)//g' {} \;
I must iterate, and reiterate, that you test this on a backup first to test it works. Otherwise, use the same command as above but change the '' in order to control the creation of backups. For example,
find . -name "*.txt" -exec sed -i '.bkp' '1 s/([^()]*)//g' {} \;
This command will perform the sed replace in the original file (keeping the filename) but will create a backup file for each with the appended .bkp
, for example test1.txt
becomes test1.txt.bkp
. This a safer option, but choose what works best for you :)
Related Topics
Get Lines of File1 Which Are Not in File2
Switch/Case Doesn't Work in Awk
Where Are Ioctl Parameters (Such as 0X1268/Blksszget) Actually Specified
Tomcat 7 with Java 8 on Windows and Linux
Vim Background with Gnu Screen
Jenkins Path to Git Windows Master/Linux Slave
Sorting Numbers with Multiple Decimals in Bash
Finding Contents of One File in Another File
Get a List of Function Names in a Shell Script
Sonar - Measure Code Coverage Using Cobertura
System Wide Keyboard Hook on X Under Linux
Equivalent Date from Gnu to Solaris
Syntax With Pound and Percent Sign After Shell Parameter Name