We have the matrices $A in mathbb R^{n times d}, W_0 in mathbb R^{d times d}$ where $d geq n$ and $rank(A) = n, rank(W_0) = d$.
$x in mathbb R^{d times 1}$ is a vector in range of $W_0^TA^T$. I want to show that it’s also in range of $W_1^TA^T$ where
$W_1 = W_0 -alpha A^TAW_0xx^T$ where $alpha in mathbb R$
What I tried:
We know $x = W_0^TA^Tv_0$, we want to find $v_1$ such that $W_1^TA^Tv_1 = W_0^TA^Tv_0$
In other words, $(W_0^T-alpha xx^TW_0^TA^TA)A^Tv_1 = W_0^TA^Tv_0$
Consider $W_0^T-alpha xx^TW_0^TA^TA$. the term on the right is a rank $1$ matrix, so by the matrix determinant lemma $det(W_0^T-alpha xx^TW_0^TA^TA) = (1 – alpha x^TW_0^TA^TAW_0^{T^{-1}}x)det(W_0^T)$
We know $det(W_0^T) neq 0$. If we assume that $alpha$ is such that the left term is non zero (we can assume such a thing) then $W_0^T-alpha xx^TW_0^TA^TA$ is invertible, so we get
$A^Tv_1 = (W_0^T-alpha xx^TW_0^TA^TA)^{-1}W_0^TA^Tv_0$
Now $A^T$ is a $d$ by $n$ matrix of rank $n$ so it has a left inverse, we can multiply by it and get:
$v_1 = A^{T^+}(W_0^T-alpha xx^TW_0^TA^TA)^{-1}W_0^TA^Tv_0$
The problem is that this operation is not invertible. I may have gotten an expression for $v_1$ but there’s no gaurantee that this $v_1$ is such that $W_1^TA^Tv_1 = x$.
Any ideas?
