Strange Return Value "134" to Call Gawk in a Bash Script

Python node2vec (Gensim Word2Vec) Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

You are almost certainly running out of memory – which causes the OS to abort your memory-using process with the SIGABRT.

In general, solving this means looking at how your code is using memory, leading up to and at the moment of failure. (The actual 'leak' of excessive bulk memory usage might, however, be arbitrarily earlier - with only the last small/proper increment triggering the error.)

Specifically with the usage of Python, and the node2vec tool which makes use of the Gensim Word2Vec class, some things to try include:

Watch a readout of the Python process size during your attempts.

Enable Python logging to at least the INFO level to see more about what's happening leading-up to the crash.

Further, be sure to:

1. Optimize your walks iterable to not compose a large in-memory list. (Gensim's Word2Vec can work on a corpus of any length, iuncluding those far larger than RAM, as long as (a) the corpus is streamed from disk via a re-iterable Python sequence; and (b) the model's number of unique word/node tokens can be modeled within RAM.)
2. Ensure the number of unique words (tokens/nodes) in your model doesn't require a model larger than RAM allows. Logging output, once enabled, will show the raw sizes involved just before the main model-allocation (which is likely failing) happens. (If it fails, either: (a) use a system with more RAM to accomdate your full set of nodes; or (b) or use a higher min_count value to discard more less-important nodes.)

If your Process finished with exit code 134 (interrupted by signal 6: SIGABRT) error does not involve Python, Gensim, & Word2Vec, you should instead:

1. Search for occurrences of that error combined with more specific details of your triggering situations - the tools/libraries and lines-of-code that create your error.
2. Look into general memory-profiling tools for your situation, to identify where (even long before the final error) your code might be consuming almost-all of the available RAM.

results of Awk command different when executed from a tty versus a script

Because command substitution (using `backquotes`) treats backquotes specially. You will have to quote every \ again. Or use $() command substitution, as suggested elsewhere.

This should work better:

endlines=`cat ${src_path}/${bakfile} |awk '/\\\\\\./ {print NR;}'`

Shell script - remove first and last quote () from a variable

There's a simpler and more efficient way, using the native shell prefix/suffix removal feature:

temp="${opt%\"}"
temp="${temp#\"}"
echo "$temp"

${opt%\"} will remove the suffix " (escaped with a backslash to prevent shell interpretation).

${temp#\"} will remove the prefix " (escaped with a backslash to prevent shell interpretation).

Another advantage is that it will remove surrounding quotes only if there are surrounding quotes.

BTW, your solution always removes the first and last character, whatever they may be (of course, I'm sure you know your data, but it's always better to be sure of what you're removing).

Using sed:

echo "$opt" | sed -e 's/^"//' -e 's/"$//'

(Improved version, as indicated by jfgagne, getting rid of echo)

sed -e 's/^"//' -e 's/"$//' <<<"$opt"

So it replaces a leading " with nothing, and a trailing " with nothing too. In the same invocation (there isn't any need to pipe and start another sed. Using -e you can have multiple text processing).

How to display every odd word using a shell script?

This works:

echo "$*" | awk '{ for (i = 1; i <= NF; i++) if (++j % 2 == 1) print $i; }'

For each field in the current line, if the incremented word number (j) is odd, print the word ($i).

Bash - Using awk to parse values from CSV then return values via ldap-search

Awk is not shell. You can't just call some UNIX program like ldapsearch directly from an awk script - that's what shell is for, awk is for manipulating text files.

You don't need/want awk or a shell loop at all for this, all you need is:

cut -d, -f8 "$csv_File" |
xargs -d '\n' -n 1 -I {} ldapsearch -Hldap://splunk.local -x -D "$admin@ldap.server" -w "$ad_Password" -b "CN={},OU=Standard,OU=People,DC=domain,DC=controller" -s sub "(cn=*)" cn useraccountcontrol'

The above uses GNU xargs for -d '\n'.

Read Effective Awk Programming, 4th Edition, by Arnold Robbins to learn awk, Shell Scripting Recipes by Chris Johnson to learn shell, and why-is-using-a-shell-loop-to-process-text-considered-bad-practice to learn more about loops in shell specifically.

Read a variable in bash with a default value

You can use parameter expansion, e.g.

read -p "Enter your name [Richard]: " name
name=${name:-Richard}
echo $name

Including the default value in the prompt between brackets is a fairly common convention

What does the :-Richard part do? From the bash manual:

${parameter:-word}
If parameter is unset or null, the expansion of word is substituted. Otherwise, the value of parameter is substituted.

Also worth noting that...

In each of the cases below, word is subject to tilde expansion, parameter expansion, command substitution, and arithmetic expansion.

So if you use webpath=${webpath:-~/httpdocs} you will get a result of /home/user/expanded/path/httpdocs not ~/httpdocs, etc.

Capturing multiple line output into a Bash variable

Actually, RESULT contains what you want — to demonstrate:

echo "$RESULT"

What you show is what you get from:

echo $RESULT

---

As noted in the comments, the difference is that (1) the double-quoted version of the variable (echo "$RESULT") preserves internal spacing of the value exactly as it is represented in the variable — newlines, tabs, multiple blanks and all — whereas (2) the unquoted version (echo $RESULT) replaces each sequence of one or more blanks, tabs and newlines with a single space. Thus (1) preserves the shape of the input variable, whereas (2) creates a potentially very long single line of output with 'words' separated by single spaces (where a 'word' is a sequence of non-whitespace characters; there needn't be any alphanumerics in any of the words).

---

Related Topics