Extracting Off-Diagonal Slice of Large Matrix

Extracting off-diagonal slice of large matrix

size <- 6
mat <- matrix(seq_len(size ^ 2), ncol = size)

low <- 0
high <- 3

delta <- rep(seq_len(ncol(mat)), nrow(mat)) - 
    rep(seq_len(nrow(mat)), each = ncol(mat))
#or Ben Bolker's better alternative
delta <- row(mat) - col(mat)
mat[delta < low | delta > high] <- NA
mat

this works with 5000 x 5000 matrices on my machine

Extracting upper off-diagonal elements of square matrix in row order

m <- matrix(1:25, 5, 5)

     [,1] [,2] [,3] [,4] [,5]
[1,]    1    6   11   16   21
[2,]    2    7   12   17   22
[3,]    3    8   13   18   23
[4,]    4    9   14   19   24
[5,]    5   10   15   20   25

t(m)[lower.tri(m)]

[1]  6 11 16 21 12 17 22 18 23 24

Extract elements from matrix diagonal saved in multiple lists in R

sapply(res, diag)

           [,1]       [,2]
[1,] 0.04770856 0.05436957
[2,] 0.03260190 0.10484454

# or
lapply(res, diag)
[[1]]
[1] 0.04770856 0.03260190

[[2]]
[1] 0.05436957 0.10484454

If you want the vectors for some reason in your global environment:

alld <- lapply(res, diag)
names(alld) <- sprintf("d.%d.el", 1:length(alld))
list2env(alld, globalenv())

Faster way of calculating off-diagonal averages in large matrices

You can get significantly faster just by extracting the diagonals directly using linear addressing: superdiag here extracts the ith superdiagonal from A (i=1 is the principal diagonal)

superdiag <- function(A,i) {
  n<-nrow(A); 
  len<-n-i+1;
  r <- 1:len; 
  c <- i:n; 
  indices<-(c-1)*n+r; 
  A[indices]
}

superdiagmeans <- function(A) {
  sapply(2:nrow(A), function(i){mean(superdiag(A,i))})
}

Running this on a 1K square matrix gives a ~800x speedup:

> A <- replicate(1000, rnorm(1000))

> system.time(sapply(1:(nrow(A)-1), function(x) mean(A[row(A) == (col(A) - x)])))
   user  system elapsed 
 26.464   3.345  29.793 

> system.time(superdiagmeans(A))
   user  system elapsed 
  0.033   0.006   0.039

This gives you results in the same order as the original.

How to split a large 640x 640 matrix up diagonally many (20-30) times over

Here is an approach using the Matrix package.

#some input
set.seed(42); m <- matrix(sample(0:1, 100, TRUE), 10)

library(Matrix)

First we define a "mask" for the slice by creating a bandsparse matrix:

mask <- bandSparse(nrow(m), ncol(m), (-2):0)
#10 x 10 sparse Matrix of class "ntCMatrix"
#                         
# [1,] | . . . . . . . . .
# [2,] | | . . . . . . . .
# [3,] | | | . . . . . . .
# [4,] . | | | . . . . . .
# [5,] . . | | | . . . . .
# [6,] . . . | | | . . . .
# [7,] . . . . | | | . . .
# [8,] . . . . . | | | . .
# [9,] . . . . . . | | | .
#[10,] . . . . . . . | | |

(-2):0 defines the off-diagonals relative to the diagonal which is 0.

Then we can apply the mask:

m1 <- m
m1[!as.matrix(mask)] <- NA
#      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
# [1,]    0   NA   NA   NA   NA   NA   NA   NA   NA    NA
# [2,]    0    1   NA   NA   NA   NA   NA   NA   NA    NA
# [3,]    0    0    0   NA   NA   NA   NA   NA   NA    NA
# [4,]   NA    1    0    1   NA   NA   NA   NA   NA    NA
# [5,]   NA   NA    0    0    1   NA   NA   NA   NA    NA
# [6,]   NA   NA   NA    1    1    1   NA   NA   NA    NA
# [7,]   NA   NA   NA   NA    1    0    1   NA   NA    NA
# [8,]   NA   NA   NA   NA   NA    1    0    0   NA    NA
# [9,]   NA   NA   NA   NA   NA   NA    1    0    0    NA
#[10,]   NA   NA   NA   NA   NA   NA   NA    0    0     1

If you don't need to distinguish 0 and NA, it becomes even easier:

as.matrix(band(m, -2, 0))
#      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
# [1,]    0    0    0    0    0    0    0    0    0     0
# [2,]    0    1    0    0    0    0    0    0    0     0
# [3,]    0    0    0    0    0    0    0    0    0     0
# [4,]    0    1    0    1    0    0    0    0    0     0
# [5,]    0    0    0    0    1    0    0    0    0     0
# [6,]    0    0    0    1    1    1    0    0    0     0
# [7,]    0    0    0    0    1    0    1    0    0     0
# [8,]    0    0    0    0    0    1    0    0    0     0
# [9,]    0    0    0    0    0    0    1    0    0     0
#[10,]    0    0    0    0    0    0    0    0    0     1

To get all slices, you only need to wrap this in a loop which iterates over diagonal numbers defining your slices.

Get all diagonal vectors from matrix

A <- matrix(1:16, 4)

# create an indicator for all diagonals in the matrix
d <- row(A) - col(A)

# use split to group on these values
split(A, d)

# 
# $`-3`
# [1] 13
# 
# $`-2`
# [1]  9 14
# 
# $`-1`
# [1]  5 10 15
# 
# $`0`
# [1]  1  6 11 16
# 
# $`1`
# [1]  2  7 12
# 
# $`2`
# [1] 3 8
# 
# $`3`
# [1] 4

Huge diaginal matrix in R

Longer answer: I suggest not creating a diagonal matrix, because in most situations you can do without it. To make that clear, consider the most typical matrix operations:

Multiply the diagonal matrix D by a vector v to produce Dv. Instead of maintaining a matrix, keep your "matrix" as a vector d of the diagonal elements, and then multiply d elementwise by v. Same result.
Invert the matrix. Again, easy: invert each element (of course, only for diagonal matrices is this generally the correct inverse).
Various decompositions/eigenvalues/determinants/trace. Again, these can all be done on the vector d.

In short, though it requires a bit of attention in your code, you can always represent a diagonal matrix as a vector, and that should solve your memory issues.

Shorter answer: Now, having said all that, of course people have already implemented the above steps implicitly using sparse matrices, which does the above steps under the hood. In R, the Matrix package is nice for sparse matrices: https://cran.r-project.org/web/packages/Matrix/Matrix.pdf

R Shiny - Extracting Anti-Diagonal elements in matrix using for-loops

Solved the issue by changing the codes to below:

for(i in 1:3) {
    for(j in 3:1) {

      my_table[,1] <- rev(my_input_matrix[i+(j-1)*3])[i] 
      my_table[,2] <- my_input_matrix[i+(j-1)*3][i]

    }
  }

How does one get the kth diagonal in R? What about the opposite diagonals?

mat = matrix(c(1:25), nrow = 5, ncol = 5, byrow = TRUE)
mat
     [,1] [,2] [,3] [,4] [,5]
[1,]    1    2    3    4    5
[2,]    6    7    8    9   10
[3,]   11   12   13   14   15
[4,]   16   17   18   19   20
[5,]   21   22   23   24   25

# Diagonal
mat[row(mat) == col(mat)]
[1]  1  7 13 19 25

# "Lower" diagonals
mat[row(mat) == col(mat)+1]
[1]  6 12 18 24
> mat[row(mat) == col(mat)+2]
[1] 11 17 23

# "Upper" diagonals
mat[row(mat) == col(mat)-1]
[1]  2  8 14 20
mat[row(mat) == col(mat)-2]
[1]  3  9 15

... but @BenBolker's answer is (of course) more elegant.

It looks like Ben deleted his answer, so I'll post a slight modification of it here. Assuming k is the number of places above the main diagonal, then:

mat[col(mat) - row(mat) == k]

will give you the diagonal k places above the main diagonal if k is positive and below if k is negative.

Per @MichaelChirico's comment, to get the "up-to-the-right" diagonals:

mat[row(mat) + col(mat) == m]

where 2 <= m <= 2*nrow(mat).

Extracting Off-Diagonal Slice of Large Matrix