Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Impact factor rankings

January 23rd, 2020, 10:13 am

 "Das ist nicht nur nicht richtig; es ist nicht einmal falsch!
Wolfgang Pauli
 
frolloos
Topic Author
Posts: 752
Joined: September 27th, 2007, 5:29 pm
Location: Netherlands

Re: Impact factor rankings

January 23rd, 2020, 10:38 am

What is not even wrong?
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Impact factor rankings

January 23rd, 2020, 3:39 pm

What is not even wrong?
"For instance selecting only particular situations under which one's proposed model performs well and not pointing out under which cirumstances it utterly fails"

e.g. "negative" probabilities, string theory, numerical schemes, life in general .. you promised me a rose garden,.
Trump's lawyer (Dershowitz?)

In Dutch, je hebt er niets aan
 
frolloos
Topic Author
Posts: 752
Joined: September 27th, 2007, 5:29 pm
Location: Netherlands

Re: Impact factor rankings

January 23rd, 2020, 4:18 pm

ah, ok, I didn't sleep much last night, everything a bit of a blur. I am not going to understand string theory today either.

Tonight's lullaby:

https://www.youtube.com/watch?v=rQnNoR0ZWOA
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Impact factor rankings

January 23rd, 2020, 6:09 pm

>but remember that it was long before science established its (to us, the only correct and obvious one) modern method - with all the good and bad (and the ugly) habits... Such "tampering" with data was quite popular in Kepler's times,

Even now there could be some tampering going on, perhaps not in the sense of blatant manipulation of data/results, but more in the 'marketing' sense. For instance selecting only particular situations under which one's proposed model performs well and not pointing out under which cirumstances it utterly fails. This is not dishonesty, and perhaps some models are too complex to be able to say exactly when it could fail, but it's not entirely in the spirit of scientific research either.
You're talking about the multiple comparison problem, or more precisely, mcp is behind this and other popular kinds of "cheating". Its two most popular instances are:
- researchers torture a sample of some objects (e.g. people) until they find some correlations of objects' properties (e.g. after testing hundreds of variables they discover that body mass is correlated with person's favourite colour). The effect would almost surely disappear if they tested for it in a new sample. It's intrinsically overfitting, and can be remedied by using a test set (or more effective approaches - see below).
- researchers can't find the effect they are looking for in one sample, so they measure a new sample, and so on, ... until they find the effect (a so-called look-elsewhere-effect, often mentioned on the occasion of LHC research)
MCP has been the source of lots of scandals it in social and medical sciences (the latest one - it got to the media, because some scientist instead of hiding with his ignorance like his colleagues, started a blog)
The methods for avoiding such errors: Tukey 1950s, Bonferoni and others 1990s, etc. - but that's too much to expect from researchers who don't bother calculating confidence intervals.
IMHO, even if they don't do this on purpose, ignorantia juris non excusat :-)

Another (meta-)problem is that nowadays science is too complex to be built on a simple popperian paradigm (via falsification). Many areas rightly require relaxing this rule: e.g. falsification is impossible nhjbb(basic science, e.g. quantum loop gravity),  a model is "worse", but faster to evaluate (technological aspect), the purpose of the model is to generate money (economic aspect), the model needs to produce nice pictures to get on the Nature cover and gain further funding (social aspect). IMHO, its OK to optimise research for techno-socio-economic aspects - as long as you don't abandon the true. The problem are pseudo-scientists who use these developments to achieve their personal goals, which have nothing to do with discovering the truth (scientific cliques, peer-review circles, etc.).

Anyway, I'm rambling on what could possibly inspire the whole new field of study.