..that 1.504 only has 3 digits accuracy and hasn't improved that much. The new quad erfc has 32 digits accuracy, so there is a big gap between the two..
Is the linear algebra part of the computation unstable and the main source of error? Are you e.g. solving a system of equations or inverting a matrix?
I'm solving a system [$]AY=B[$] by row reduction rather than inverting [$]A[$]
the main source of error is I have an infinite sum which I am truncating
I have [$]\sum_{n=-\infty}^{\infty}f_{n}(x)y_{n}=b(x)[$] where the [$]f_{n}(x)[$] are (known) functions and the [$]y_{n}[$] are (unknown) coefficients. This equation is true for all [$]x[$] in some range
I truncate the series and evaluate at the [$]2N+1[$] gridpoints [$]x_{m}[$]
[$]\sum_{n=-N}^{N}f_{n}(x_{m})y_{n}=b(x_{m})[$]
and write [$]A_{mn}=f_{n}(x_{m})[$] and [$]B_{m}=b(x_{m})[$] gives [$]\sum_{n=-N}^{N}A_{mn}y_{n}=B_{m}[$]
The larger [$]N[$] is, the smaller the error, but there's a very flat spectrum: when I solve the truncated system and plot [$]y_{n}[$] against [$]n[$], the slope is very gradual