How to Export Daily Disk Usage to CSV Format in Shell Scripting

How to export daily disk usage to csv format in shell scripting?

Assuming you want all filesystems you can simplify that to:

printf '%s\n' "$(date +%Z-%Y-%m-%d_%H-%M-%S)" >> excel.csv
df -h | awk '{print $1","$2","$3","$4","$5","$6}' >> excel.csv

How to export daily disk space usage?

Use awk to parse df result:

Assuming ; is your csv separator, then:

df -h | awk 'FNR == 2 {print $2";"$3";"$4}'

gives:

24G;5.4G;18G

In a full script with the date:

#!/bin/bash

SEPARATOR=","
SIZES=`df -h | awk -v SEP="$SEPARATOR" 'FNR == 2 {print SEP$2SEP$3SEP$4}'`
echo `date +%Z-%Y-%m-%d_%H-%M-%S`"$SIZES" >> test.csv

disk space info to csv with date using cron job

/var/lib/psgl96 isn't a command, it's a directory. You need to use the cd command to switch to it.

*/1 * * * * cd /var/lib/pgsql96; { date +\%F_\%H:\%M:\%S; df -h | tr -s ' ' ','; } > diskspaceinfo.csv

How to export a PostgreSQL query output to a csv file

Modern syntax:

COPY (SELECT * FROM ...) TO '/tmp/filename.csv' (FORMAT csv);

So the 162 rows of my output table have been copied in the shell. How
can I paste or move them to a csv file?

The result is the CSV file. Open it with any spreadsheet program using matching delimiters. The manual:

The default is a tab character in text format, a comma in CSV format

The psql meta command \copy is a wrapper around the SQL COPY function. It writes and reads files local to the client (while COPY uses files local to the server) and does not require superuser privileges.

See:

• Export specific rows from a PostgreSQL table as INSERT SQL script
• PostgreSQL: export resulting data from SQL query to Excel/CSV

How to extract one column of a csv file

You could use awk for this. Change '$2' to the nth column you want.

awk -F "\"*,\"*" '{print $2}' textfile.csv

Script to send emails with text extracted from CSV file

You could do something like this:

#!/bin/bash

#get line count
linecount=$(cat mycsv.csv  | wc -l)
#set counting variable
counter=1
#loop through file
while read line; do 
    #split each line into an array by comma
    IFS=, read -a arr <<< "$line"; 
    #set  array values to variables
    subject="${arr[0]}"; 
    intro="${arr[1]}"; 
    details="${arr[2]}"; 
    closing="${arr[3]}"; 
    #create body
    body=$(printf "Intro: %s \n Details: %s \n Closing: %s" "$intro" "$details" "$closing")
    #echo progress
    echo "Sending email $counter of $linecount: Subject: $subject, Body: $body"
    #send email
    echo "$body" | mail -s "$subject" recipient@email.com 
    #advance counter
    counter=$(( $counter +1 ))
    #sleep for a second
    sleep 1
done < mycsv.csv 

The above would output:

Sending email 1 of 2: Subject: subject, Body: Intro: intro 
 Details: details 
 Closing: closing

You could also format the body of the email this way. Note if you do this the second EOF that closes the email body must all the way to the left side of the file with no white space between the beginning of the line and the EOF.

    #send email
    mail -s "$subject" recipient@email.com <<EOF
    "$intro"
    "$details"
    "$closing"
EOF

Where mycsv.csv looks like:

subject,intro,details,closing
subject1,intro1,details1,closing1

I don't have access to OSX so this was tested with GNU bash, version 4.3.42:

bash --version
GNU bash, version 4.3.42(1)-release (x86_64-redhat-linux-gnu)

formatting data in csv format with awk

The following produces TSV (tab-separated) output matching your desired output (it won't align perfectly when printed in the console, but Excel should be able to read it with fields separated as intended):

awk '
  NF==1 { printf "%s", $1; next }
  { 
    printf "\t%s", $1
    sep="\t"
    for (i=3; i<=NF; ++i) {
      gsub("[=/()]", "\t", $i)
      printf "%s%s", sep, $i
      sep="\n\t\t"
    }
    printf "\n"
  }
' 11.csv

Shell script to copy files from one location to another location and rename add the current date to every file

In bash, provided you files names have no spaces:

cd /home/webapps/project1/folder1
for f in *.csv
do 
   cp -v "$f" /home/webapps/project1/folder2/"${f%.csv}"$(date +%m%d%y).csv
done

SFTP bash shell script to copy the file from source to destination

In a simple case such as this, you could use scp instad of sftp and specify the files to copy on the command line:

 scp $localpath/* username@10.42.255.209:/$remotepath/

But if you would rather want to issue sftp commands, then sftp can read commands from its stdin, so you can do:

  echo "put $localpath/* $remotepath" | sftp username@10.42.255.209

Or you can use a here document to pass data as stdin to sftp, which might be easier if you want to run several sftp commands:

sftp username@10.42.255.209 << EOF
put $localpath/fileA $remotepath/
put $localpath/fileB $remotepath/
EOF

Finally, you could place the sftp commands in a separate file, say sftp_commands.txt , and have sftp execute those commands using its -b flag:

 sftp -b ./sftp_commands.txt username@10.42.255.209

---

Related Topics