Curl Progress Bar: How to Pipe and Extract Numbers Only Using Grep

CURL Progress Bar: How to pipe and extract numbers only using grep?

You can't get the progress info like that through grep; it doesn't make sense.

curl writes the progress bar to stderr, so you have to redirect to stdout before you can grep it:

$ curl -# -o f1.flv 'http://osr.com/f1.flv' 2>&1 | grep 1 | less results in:

^M                                                                           0.0
%^M######################################################################## 100.
0%^M######################################################################## 100
.0%^M######################################################################## 10
0.0%

Are you expecting a continual stream of numbers that you are redirecting somewhere else? Or do you expect to grab the numbers at a single point?

If it's the former, this sort of half-assedly works on a small file:

$ curl -# -o f1.flv 'http://osr.com/f1.flv' 2>&1 | sed  's/#//g' -
 100.0%                                                                    0.0%

But it's useless on a large file. The output doesn't print until the download is finished, probably because curl seems to be sending ^H's to the terminal. There might be a better way to sed it, but I wouldn't hold my breath.

$ curl -# -o l.tbz 'ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2009/06/2009-06-02-05-mozilla-1.9.1/firefox-3.5pre.en-US.linux-x86_64.tar.bz2' 2>&1 | sed 's/#//g' -
 100.0%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Curl progress meter and bar problem. It duplicate data

Regarding your question

How I can show only percentage of progress?

this seems not to be possible easily according CURL Progress Bar: How to pipe and extract numbers only using grep? or without a wrapper according curl progress - only show percentage.

extract token from curl result by shell script

Instead of using the direct result of cURL, you could save the result in a file, and use your grep command on it.
Something like this maybe :

curl -o boulou.txt http://localhost/identity/v3/auth/tokens && cat boulou.txt | grep "X-Subject-Token" | awk '{printf $2}'

Edit, if you just want you desired output, add the --silent to the cURL command :

curl -o boulou.txt http://localhost/identity/v3/auth/tokens --silent && cat boulou.txt | grep "X-Subject-Token" | awk '{printf $2}'

Edit 2: If you want to export it, and delete your file, you could use something like this :

export OS_TOKEN=$(curl -o billy.txt hhttp://localhost/identity/v3/auth/tokens --silent &&  cat billy.txt | grep "X-Subject-Token" | awk '{printf $2}') && rm billy.txt

Assign output to variable in Bash

In shell, you don't put a $ in front of a variable you're assigning. You only use $IP when you're referring to the variable.

#!/bin/bash

IP=$(curl automation.whatismyip.com/n09230945.asp)

echo "$IP"

sed "s/IP/$IP/" nsupdate.txt | nsupdate

How do I get cURL to not show the progress bar?

curl -s http://google.com > temp.html

works for curl version 7.19.5 on Ubuntu 9.10 (no progress bar). But if for some reason that does not work on your platform, you could always redirect stderr to /dev/null:

curl  http://google.com 2>/dev/null > temp.html

How to extract only the raw contents of an ELF section?

Rather inelegant hack around objdump and dd:

IN_F=/bin/echo
OUT_F=./tmp1.bin
SECTION=.text

objdump -h $IN_F |
  grep $SECTION |
  awk '{print "dd if='$IN_F' of='$OUT_F' bs=1 count=$[0x" $3 "] skip=$[0x" $6 "]"}' |
  bash

The objdump -h produces predictable output which contains section offset in the elf file. I made the awk to generate a dd command for the shell, since dd doesn't support hexadecimal numbers. And fed the command to shell.

In past I did all that manually, without making any scripts, since it is rarely needed.

How to check a column contain non numeric values

awk -F'|' '{print $1}' File | awk '/^[0-9]$/' | wc -l

How to extract one column of a csv file

You could use awk for this. Change '$2' to the nth column you want.

awk -F "\"*,\"*" '{print $2}' textfile.csv

---

Related Topics