1. Selecting the category of the approximating curve (e.g., a global polynomial, chain of splines, sinusoids, wavelets, multi-population model with independent curves, etc., etc
So, what's the answer? What are the criteria that lead to our choosing method 1 over method 2?
Picking a curve fit method seems to involve a combination of:
1) Metadata: Knowledge about the source (or application) of the data might include knowledge of likely mathematical properties such as form, derivatives, continuity, etc. of the underlying data generation phenomenon. That might lead to selecting a global versus a local curve fit (e.g., fitting a piece-wise description of a road network to GPS data points).
2) Heuristic Evaluation: eye-balling the shape of the scattergram for evidence of locality, periodicity, nonlinearity, etc.
3) Computational cost: Development expediency (e.g., using Excel's polynomial curve fit) or data throughput issues (e.g., using discrete cosines to fit image data) might drive curve fit method selection.
4) Minimizing an Error cost function: One might try multiple methods and pick the one with the least error (as "error" is defined for the problem).
Let me try in this way by a question: what is the rationale/reason for using global polynomials in AI? I did look in a few books but no reason was given. I suppose someone can come out and give an answer.
Global polynomials are not used in most applications of numerical analysis these days AFAIK. It is pre-1960s technique. In fairness, maybe AI is dealing with other issues.
That remark about using a 300-degree polynomial was hilarious. I hope you were joking. Was the example taken from Geron's book?
The probable reason for using global polynomials is an appeal to Taylor series and their ability to fit a great many types of functions (but not all!); computational simplicity; internal encoding of the derivatives of the curve; and familiarity to those who know basic maths (algebra & calculus).
Is a 300-degree polynomial any more ridiculous than using a million or more sine-wave segments?