How to Show Dialog Gauge for Wget

How to show dialog gauge for wget?

URL="http://upload.wikimedia.org/wikipedia/commons/4/4e/Pleiades_large.jpg"
wget "$URL" 2>&1 | \
 stdbuf -o0 awk '/[.] +[0-9][0-9]?[0-9]?%/ { print substr($0,63,3) }' | \
 dialog --gauge "Download Test" 10 100

How can I show the wget progress bar only?

You can use the following filter:

progressfilt ()
{
    local flag=false c count cr=$'\r' nl=$'\n'
    while IFS='' read -d '' -rn 1 c
    do
        if $flag
        then
            printf '%s' "$c"
        else
            if [[ $c != $cr && $c != $nl ]]
            then
                count=0
            else
                ((count++))
                if ((count > 1))
                then
                    flag=true
                fi
            fi
        fi
    done
}

Usage:

$ wget --progress=bar:force http://somesite.com/TheFile.jpeg 2>&1 | progressfilt
100%[======================================>] 15,790      48.8K/s   in 0.3s

2011-01-13 22:09:59 (48.8 KB/s) - 'TheFile.jpeg' saved [15790/15790]

This function depends on a sequence of 0x0d0x0a0x0d0x0a0x0d being sent right before the progress bar is started. This behavior may be implementation dependent.

bash - getting a number from a stream

Progress bar with wget, whiptail and GNU sed:

wget --progress=dot 'URL' 2>&1 | sed -un 's/.* \([0-9]\+\)% .*/\1/p' | whiptail --gauge "Download" 7 50 0

Using Bash to display a progress indicator

In this example using SCP, I'm demonstrating how to grab the process id (pid) and then do something while that process is running.

This displays a simple spinnng icon.

/usr/bin/scp me@website.com:file somewhere 2>/dev/null &
pid=$! # Process Id of the previous running command

spin[0]="-"
spin[1]="\\"
spin[2]="|"
spin[3]="/"

echo -n "[copying] ${spin[0]}"
while [ kill -0 $pid ]
do
  for i in "${spin[@]}"
  do
        echo -ne "\b$i"
        sleep 0.1
  done
done

---

William Pursell's solution

/usr/bin/scp me@website.com:file somewhere 2>/dev/null &
pid=$! # Process Id of the previous running command

spin='-\|/'

i=0
while kill -0 $pid 2>/dev/null
do
  i=$(( (i+1) %4 ))
  printf "\r${spin:$i:1}"
  sleep .1
done

How to add a progress bar to a shell script?

You can implement this by overwriting a line. Use \r to go back to the beginning of the line without writing \n to the terminal.

Write \n when you're done to advance the line.

Use echo -ne to:

1. not print \n and
2. to recognize escape sequences like \r.

Here's a demo:

echo -ne '#####                     (33%)\r'
sleep 1
echo -ne '#############             (66%)\r'
sleep 1
echo -ne '#######################   (100%)\r'
echo -ne '\n'

In a comment below, puk mentions this "fails" if you start with a long line and then want to write a short line: In this case, you'll need to overwrite the length of the long line (e.g., with spaces).

Get git clone progress for dialog --gauge

As I think my comment is really too long and unclear here is my udnerstanding:

git clone git@github.com:really-bit-git-repo output-dir --progress 2>&1 | cat > /tmp/gitprocfile &

Calling git and writing the output in a file, cat is not necessary here.

cat /tmp/gitprocfile | grep "[0-9]\{1,2\}%" | awk '{print $7}' | dialog --gauge "Progress" 7 50 

Just after the prvious command, show the file one with cat and dot the grep etc. So you get the content of the file only once ...

If wanting a text output at end I would do:

 git clone git@github.com:really-bit-git-repo output-dir --progress 2>&1 | tee /tmp/gitprocfile | grep "[0-9]\{1,2\}%" | awk '{print $7}' | dialog --gauge "Progress" 7 50

If the dialog is in antoher term than the gitcommand:

In terminal 1:

echo '' > /tmp/gitprocfile; git clone git@github.com:really-bit-git-repo output-dir --progress 2>&1 >> /tmp/gitprocfile &

In terminal 2:

tail -f /tmp/gitprocfile | grep "[0-9]\{1,2\}%" | awk '{print $7}' | dialog --gauge "Progress" 7 50 

I'm not used to dialog, but at least I'm sure your attempt won't reload the file on change.

CURL Progress Bar: How to pipe and extract numbers only using grep?

You can't get the progress info like that through grep; it doesn't make sense.

curl writes the progress bar to stderr, so you have to redirect to stdout before you can grep it:

$ curl -# -o f1.flv 'http://osr.com/f1.flv' 2>&1 | grep 1 | less results in:

^M                                                                           0.0
%^M######################################################################## 100.
0%^M######################################################################## 100
.0%^M######################################################################## 10
0.0%

Are you expecting a continual stream of numbers that you are redirecting somewhere else? Or do you expect to grab the numbers at a single point?

If it's the former, this sort of half-assedly works on a small file:

$ curl -# -o f1.flv 'http://osr.com/f1.flv' 2>&1 | sed  's/#//g' -
 100.0%                                                                    0.0%

But it's useless on a large file. The output doesn't print until the download is finished, probably because curl seems to be sending ^H's to the terminal. There might be a better way to sed it, but I wouldn't hold my breath.

$ curl -# -o l.tbz 'ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2009/06/2009-06-02-05-mozilla-1.9.1/firefox-3.5pre.en-US.linux-x86_64.tar.bz2' 2>&1 | sed 's/#//g' -
 100.0%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

cURL: How to display progress information while uploading?

This is what I use in one of my build scripts:

curl "${UPLOAD_URL}" \
    --progress-bar \
    --verbose \
    -F build="${BUILD}" \
    -F version="${VERSION}" \
    -F ipa="@${IPA};type=application/octet-stream" \
    -F assets="@-;type=text/xml" \
    -F replace="${REPLACE}" \
    -A "${CURL_FAKE_USER_AGENT}" \
    <<< "${ASSETS}" \
    | tee -a "${LOG_FILE}" ; test ${PIPESTATUS[0]} -eq 0

The -F and -A options will probably not be of interest to you, but the helpful parts are:

curl "${UPLOAD_URL}" --progress-bar

which tells curl to show a progress bar (instead of the default 'progress meter') during the upload, and:

 | tee -a "${LOG_FILE}" ; test ${PIPESTATUS[0]} -eq 0

which appends the output of the command to a log file and also echo's it to stdout. The test ${PIPESTATUS[0]} -eq 0 part makes it so that the exit status of this line (which is in a bash script) is the same exit code that the curl command returned and not the exit status of the tee command (necessary because tee is actually the last command being executed in this line, not curl).

---

From man curl:

PROGRESS METER
       curl normally displays a progress meter during operations, indicating the
       amount of transferred data, transfer speeds and estimated time left, etc.

       curl  displays  this  data to the terminal by default, so if you invoke curl
       to do an operation and it is about to write data to the terminal, it disables
       the progress meter as otherwise it would mess up the output mixing progress
       meter and response data.

       If you want a progress meter for HTTP POST or PUT requests, you need to
       redirect the response output to a file, using shell redirect (>), -o [file]
       or similar.

       It is not the same case for FTP upload as that operation does not spit out
       any response data to the terminal.

       If you prefer a progress "bar" instead of the regular meter, -# is your
       friend.

OPTIONS
       -#, --progress-bar
              Make curl display progress as a simple progress bar instead of the
              standard, more informational, meter.

---

Related Topics