Remove Columns from Dataframe Where All Values Are Na

Remove columns from dataframe where ALL values are NA

Try this:

df <- df[,colSums(is.na(df))<nrow(df)]

How to delete columns that contain ONLY NAs?

One way of doing it:

df[, colSums(is.na(df)) != nrow(df)]

If the count of NAs in a column is equal to the number of rows, it must be entirely NA.

Or similarly

df[colSums(!is.na(df)) > 0]

Pandas: drop columns with all NaN's

From the dropna docstring:

Drop the columns where all elements are NaN:

df.dropna(axis=1, how='all')


   A    B    D
0  NaN  2.0  0
1  3.0  4.0  1
2  NaN  NaN  5

Remove columns that are all NA for at least one level of a factor

Another option:

 dat %>%
  select(site, dat %>%
  group_by(site) %>%
  summarise(across(everything(), ~!all(is.na(.x))))%>%
  ungroup() %>%
  select(-site) %>%
  select(where(all))%>%
  names())

   site year species_B species_C
1     A 2000         1         1
2     A 2001         2         2
3     A 2002        NA         3
4     A 2003         4         4
5     A 2004         5         5
6     B 2000        NA         2
7     B 2001         3         3
8     B 2002         4         4
9     B 2003         5         5
10    B 2004         6         6

Remove columns from dataframe where some of values are NA

The data:

Itun <- data.frame(v1 = c(1,1,2,1,2,1), v2 = c(NA, 1, 2, 1, 2, NA)) 

This will remove all columns containing at least one NA:

Itun[ , colSums(is.na(Itun)) == 0]

An alternative way is to use apply:

Itun[ , apply(Itun, 2, function(x) !any(is.na(x)))]

removing columns with NA values only

The tidyverse approach would look like this (also using @Rich Scriven data):

d %>% select_if(~any(!is.na(.)))
#    x
# 1 NA
# 2  3
# 3 NA

Remove Column if all values are either NA or 0 in r

Multiple ways to do this

df[colSums(is.na(df) | df == 0) != nrow(df)]

#  rv X1  X2 X4
#1  M  0 110  1
#2  J 70 200  3
#3  J NA 115  4
#4  M 65 110  9
#5  J 70 200  3
#6  J 64 115  8

Using apply

df[!apply(is.na(df) | df == 0, 2, all)]

Or using dplyr

library(dplyr)
df %>%  select_if(~!all(is.na(.) | . == 0))

Remove columns from dataframe where ALL values are NA, NULL or empty

We can use Filter

Filter(function(x) !(all(x=="")), df)
#   Var1 Var3
#1  2R+   52
#2  2R+  169
#3  2R+   83
#4  2R+   98
#5  2R+   NA
#6  2R+  111
#7  2R+   94
#8  2R+  116
#9  2R+   86

NOTE: It should also work if all the elements are NA for a particular column

df$Var3 <- NA
Filter(function(x) !(all(x=="")), df)
#   Var1
#1  2R+
#2  2R+
#3  2R+
#4  2R+
#5  2R+
#6  2R+
#7  2R+
#8  2R+
#9  2R+

Update

Based on the updated dataset, if we need to remove the columns with only 0 values, then change the code to

Filter(function(x) !(all(x==""|x==0)), df2)
#    VAR1 VAR3 VAR4 VAR7
#1  2R+   52 1.05   30
#2  2R+  169 1.02   40
#3  2R+   83   NA   40
#4  2R+   98 1.16   40
#5  2R+  154 1.11   40
#6  2R+  111   NA   15

data

df2 <- structure(list(VAR1 = c("2R+", "2R+", "2R+", "2R+", "2R+", "2R+"
), VAR2 = c("", "", "", "", "", ""), VAR3 = c(52L, 169L, 83L, 
98L, 154L, 111L), VAR4 = c(1.05, 1.02, NA, 1.16, 1.11, NA), VAR5 = c(0L, 
0L, 0L, 0L, 0L, 0L), VAR6 = c(0L, 0L, 0L, 0L, 0L, 0L), VAR7 = c(30L, 
40L, 40L, 40L, 40L, 15L)), .Names = c("VAR1", "VAR2", "VAR3", 
"VAR4", "VAR5", "VAR6", "VAR7"), row.names = c("1", "2", "3", 
"4", "5", "6"), class = "data.frame")

---

Related Topics