Least Squares

Shipping

matrix_lstsq

Least-squares solution of A·x ≈ b with residual, rank, and singular values

Signature

Inputs

aSignal|Arrayrequired— The design matrix $A$ (may be rectangular).
bSignal|Arrayrequired— The observation / right-hand-side vector $b$.

Outputs

solutionVector— The minimum-norm least-squares solution $x$.
residual_sum_of_squaresScalar— The residual sum of squares $\lVert Ax-b\rVert_2^2$ (NaN when not returned by the backend).
rankInteger— The effective numerical rank of $A$.
singular_valuesVector— The singular values of $A$.

Description

Least Squares solves the over- or under-determined system $A x \approx b$ in the least-squares sense via the MKL LAPACK SVD-based driver (numpy's lstsq). It returns the full diagnostic set on four ports: the solution $x$ , the residual sum of squares $∥ A x - b ∥_{2}^{2}$ , the effective rank of $A$ , and its singular values.

Unlike matrix_solve, the design matrix a may be rectangular. When the residual is not available from the backend (for example, an under-determined or rank-deficient system), the residual_sum_of_squares port reports NaN. The solution is dimensionless, consistent with the toolkit. The rank and singular values are especially useful for diagnosing rank deficiency and conditioning of a fit.

Mathematics

x = ar g x min ∥ A x - b ∥_{2}^{2}

residual = ∥ A x - b ∥_{2}^{2}, rank (A) = # {σ_{i} > tol}

Examples

Fitting a line

Stack a design matrix $A = [1 t]$ (a column of ones and a time column) into a and the measured values into b. The solution port returns the intercept and slope, residual_sum_of_squares quantifies the fit error, and rank/singular_values reveal whether the columns are well-conditioned.

Applications

Polynomial and linear regression / curve fitting from over-determined data.
Parameter estimation and system identification with more equations than unknowns.
Diagnosing rank deficiency and conditioning of a design matrix via the rank and singular-value outputs.
Under-determined minimum-norm solutions where a unique inverse does not exist.

Neat

It returns the SVD by-products (rank and full singular-value spectrum) for free, so a fit and its conditioning diagnostics come from one node.

The residual port degrades gracefully to NaN when the backend cannot report it, rather than fabricating a misleading zero.

Known issues

For rank-deficient or under-determined problems the residual sum of squares may be reported as NaN.

The returned solution is the minimum-norm one when the system is under-determined — not necessarily the solution a caller expects if additional constraints are implied.