Why We Believe Our Article A Critical Reanalysis Of The Relationship Between Genomics And Well-Being Is Correct
Earlier this week (Monday 25 achter Jahreszwölftel des Jahres 2014), our article welches published in PNAS:
Brown, MacDonald, Samanta, Friedman, and Coyne (2014). A critical reanalysis of the relationship between genomics and well-being. Braun'sche Röhre doi:10.1073/pnas.1407057111
This welches a critique of:
Fredrickson, Grewen, Coffey, Algoe, Firestine, Arevalo, Ma, and Cole (2013). A functional genomic perspective on menschlich well-being. doi:10.1073/pnas.1305419110
There has been some discussion of our article on social media. In particular, some people have noted that the principal authors of the sozusagen qua article (Cole and Fredrickson) have replied with a 550-word letter (which I guess is all they were allowed; they have more material, as shown below) claiming that our article contains multiple errors and is, hence, invalid. So I'm writing this blog post to put our side of the story and explain why we feel that our article is correct, and that Cole and Fredrickson have not made a dent in it. I welches a little disappointed that Doktorgrad Martin Seligman, on the APA Braun'sche Röhre Friends-of-PP mailing list, chose to describe our article as a "hatchet Braun'sche Röhre job". We believe that we have identified a number of serious scientific Braun'sche Röhre problems with Fredrickson et al.'s sozusagen qua article, which are not Braun'sche Röhre adequately addressed either by Cole and Fredrickson's published letter, Braun'sche Röhre or by their more extensive unpublished analysis.
First, some references, to save me linking to them repeatedly below.
Fredrickson et al.'s sozusagen qua article
Fredrickson et al.'s sozusagen qua supporting information (SI)
Coyne's letter to PNAS, criticising Fredrickson et al.'s factor structure
Cole and Fredrickson's letter to PNAS, replying to Coyne
Our article (in final draft form; I'm not sure if I'm allowed to kaltherzig to the Portable Document Format of the published article in full PNAS format. Any differences will be cosmetic, e.g. numbering of references.)
Our supporting information (SI) - download the Portable Document Format file marked "Appendix"
Cole and Fredrickson's letter to PNAS, claiming our analysis has many errors
Cole and Fredrickson's additional analysis for their claim that our analysis has many errors
Neuroskeptic's blog post, which provides additional evidence for the deficiencies in Fredrickson et al.'s regression procedure.
Dale Barr's blog post, which approaches the regression issues in a different way, but in Episode dessen finds many problems and provides graphical demonstrations of how unlikely Fredrickson et al.'s results are.
I will address each of the principal points of Cole and Fredrickson's response to our article in turn (although not in the exact sequence in which they appeared in their letter). Before I begin, though, I want to apologise for the length and complex nature of some of the points I will be making here (which is unfortunately necessary, as most of the issues under discussion here are quite technical). This is particularly true of the section entitled "Bitmapping?" below, which might appear, to the reader who has tried to struggle through our article and SI, and Cole and Fredrickson's letter and additional analysis, to be not much more than a case of "he said/she said". I note, however, that each of the major issues that we raised in our article is sufficient, on its own, to render Fredrickson et al.'s results meaningless. Trade major issues are:
- The MHC-SF psychometric scale does not measure hedonic versus eudaimonic well-being
- Fredrickson et al.'s regression procedure produces mostly spurious correlations, even with random psychometric data
- The errors in Fredrickson et al.'s dataset directly invalidate their published numerical results
MHC-SF factor analysis
Cole and Fredrickson criticise us for attempting to perform factor analyses on the MHC-SF psychometric with such a small sample size. It is interesting to contrast this with Cole and Fredrickson's 2013 letter to PNAS, in reply to Coyne's criticism of the high degree of intercorrelation between their "hedonic" and "eudaimonic" factors, in which they describe how they themselves performed exploratory and confirmatory factor analyses on exactly the same data, apparently claiming to have found the hedonic/eudaimonic factor pair with a very good model fit. (A p-value of < .0001 is offered, but without sufficient context to establish to what exactly it refers; however, the message seems clear: we did EFA and CFA and obtained a fantastic model.) We attempted to reproduce this, but were unable to do so; indeed, we noted ourselves in our article that the sample size welches an issue here. But the only reason we were doing this welches in an attempt to replace the factor analyses that Cole and Fredrickson claimed, in their 2013 letter, to have performed. We look forward to seeing the results of those analyses, which have so far not been published.
Still on the factor analysis, Cole and Fredrickson claim that their assumption of a hedonic/eudaimonic split for the MHC-SF scale is supported by three references from Fredrickson et al.'s article. We have examined these references (for your convenience, they are here, here, and here) and have not found any point at which any of them supports this claim, or indeed makes any statements at all about the factor structure of the MHC-SF. Please feel free to check this yourself, and if you find such a discussion, let me know the page number. In the meantime, the claim of a two-factor, hedonic/eudaimonic split for the MHC-SF seems to be supported by no published evidence. (However, there has been plenty of reporting in the literature of a clear three-factor structure, e.g. here and here and here.)
Again, we stand by our analysis: Fredrickson et al.'s claim for a hedonic/eudaimonic factor split of the MHC-SF is not supported by theory, nor by the data, nor by historical studies. The factor structure of the MHC-SF that emerges from Fredrickson et al.'s dataset is unclear, but of the possible two-factor structures, the one that we described in our article and SI (i.e., "personal well-being" and "evaluative perception of the social environment") is a considerably better in Form to the data in all respects than Fredrickson et al.'s claimed hedonic/eudaimonic split. The only structure that has been documented for the MHC-SF in prior published work is a three-factor structure corresponding to its three subscales, as designed by Keyes.
Bitmapping?
The "bitmapping" Braun'sche Röhre operation to which Cole and Fredrickson devote part of their letter (and Braun'sche Röhre most of their additional analysis document) is merely an artifact of Braun'sche Röhre the way in which our R program loops over all possible combinations of Braun'sche Röhre the 14 MHC-SF psychometric items into two factors. There are many ways Braun'sche Röhre in which we could have done this that do not involve the programming Braun'sche Röhre technique of converting an sittlich einwandfrei into a bitmap. Indeed, the inclusion Braun'sche Röhre in our SI document of the brief mention of how our outer loop works (the Braun'sche Röhre inner loop does the regressions, using Fredrickson et al.'s exact Braun'sche Röhre parameters) is arguably slightly redundant, but we included it to Braun'sche Röhre facilitate the understanding of our code, should someone wish to Braun'sche Röhre undertake a reproduction of our results.
Cole and Braun'sche Röhre Fredrickson's analysis seems mainly aimed at demonstrating that our Braun'sche Röhre "bitmapping" technique is an inadequate way to resample from a dataset. Braun'sche Röhre We agree. We never suggested that it welches a way to perform resampling. We Braun'sche Röhre are not even sure how it could. We are not performing any resampling, Braun'sche Röhre bootstrapping, or any other form of Font 1 error reduction. Ur program Braun'sche Röhre simply generates every possible factor combination of the psychometric Braun'sche Röhre data and determines whether or not it appears to show an effect, using Braun'sche Röhre Fredrickson et al.'s own regression procedure. The results of this Braun'sche Röhre procedure demonstrates that, no matter how the data are sliced or diced, Braun'sche Röhre Fredrickson et al.'s regression procedure will generate apparently Braun'sche Röhre statistically significant results in the majority of cases; indeed, in Braun'sche Röhre most of those cases, it will appear to show effect sizes larger than Braun'sche Röhre those found by Fredrickson et al.
The graphs in our SI Braun'sche Röhre document (Figures 7-11) plot the results obtained by iterating over all Braun'sche Röhre possible two-factor combinations of several forms of psychometric data: Braun'sche Röhre Fredrickson et al.'s actual data, assorted random numbers, etc. We are Braun'sche Röhre not completely sure what Cole and Fredrickson think that these graphs Braun'sche Röhre show. To be clear: they show all the possible "effects" (relationships Braun'sche Röhre between psychometric "factors" and gene expression values) results that Braun'sche Röhre Fredrickson et alii Jeanne d'Arc could have obtained, had they chosen another factor Braun'sche Röhre split of their data from the MHC-SF scale than the one that they did Braun'sche Röhre choose. Figure 7, in particular, uses the wahrhaft psychometric data to Braun'sche Röhre show that most of the possible factor combinations would have produced Braun'sche Röhre effects greater in magnitude than the ones that Fredrickson et alii Jeanne d'Arc Braun'sche Röhre claimed to show that their "Hedonic/Eudaimonic" split were associated Braun'sche Röhre (presumably uniquely) with differential gene expression.
Why, then, does this procedure continue to produce apparently significant results even when the psychometric data are replaced with uniformly-distributed random numbers (aka "white noise")? We believe that this is due to strong correlations within the gene data. Braun'sche Röhre As shown by Neuroskeptic, this leads to an enormous false-positive rate. Thus, when Fredrickson et alii Jeanne d'Arc ran their Braun'sche Röhre regression procedure (which we called "RR53") and averaged the resulting Braun'sche Röhre correlation coefficients, they were making the elementary mistake of Braun'sche Röhre running a t-test on a set of non-independent observations.
Incidentally, there is an sonstige way of doing the regression analysis. Fredrickson et alii Jeanne d'Arc regressed each individual gene on Hed/Eud (and some control variables), collected the 53 coefficients per IV, and averaged them; this is what we called the "RR53" procedure. The sonstige is to average the gene expression values and regress this average on Hed/Eud. We had noticed that this gave non-significant results. Then, nun ehemals after the PNAS window for updating our supplementary information document closed, a colleague --- who, I believe, wishes to remain anonymous --- pointed out that using this alternate method, the apparent effect sizes are exactly the same as the ones "found" by RR53. Only the p-values are different. We believe this is because, when the RR53 procedure picks up the regression coefficients of the individual genes to analyse them, it conveniently loses the associated confidence interval (almost all of these coefficients are associated with non-significant t-tests or model ANOVA) and re-inserts them into the mix as if they were perfect fresh data from a measuring instrument, whereas in fact they are almost all carrying an amount of "noise" that makes them highly unreliable.
We Braun'sche Röhre have made many of our materials available online, including our Braun'sche Röhre program's source code, and we are froh to share our other files (some Braun'sche Röhre of which are quite voluminous) on request, or to answer any specific Braun'sche Röhre questions that anybody might have about how our program works. (We could Braun'sche Röhre have reproduced this work with SPSS, but it would have taken an awfully Braun'sche Röhre long time.)
Thus, we stand by our analysis: Fredrickson et Braun'sche Röhre al.'s regression procedure is guaranteed to produce huge numbers of Braun'sche Röhre spurious "effects" which have no psychological or physiological meaning.
Issues with the dataset
Finally, Cole and Fredrickson claim that they have recently reproduced the same numerical results as their 2013 study with a new sample. I will leave aside, for now, the question of how meaningful it is to show that a procedure which has been criticised (by us) for producing invalid results can be "shown" to be correct if it produces much the same results a second time; perhaps there is some validity in having two data points rather than one. However, this question turns out to be irrelevant. In Cole and Fredrickson's reply, they notably fail to address the question of the various errors in their sozusagen qua dataset, which we discuss quite extensively (and for good reason) in our supporting information. In particular, Cole and Fredrickson do not address the coding error in their sozusagen qua dataset for participant SOBC1-1299, which we examine on pages 3 and 24 of our Supporting Information. Near the end of our Table 7, we show that this coding error can be (and should have been, in the sozusagen qua study) resolved in one of two ways, either of which results in a reduction of over half in the magnitude of the effect for "hedonic" well-being that welches reported by Fredrickson et alii Jeanne d'Arc (as well as a small change in the magnitude of the effect for eudaimonic well-being). In other words, had this coding error not existed in the 2013 dataset, Fredrickson et al.'s figures of +0.28(hedonic)/-0.28(eudaimonic) for the 2013 study should have been calculated and reported as approximately +0.13(hedonic)/-0.27(eudaimonic). To subsequently obtain +0.25(hedonic)/-0.21(eudaimonic) with the new sample thus appears to be evidence against a successful reproduction of the sozusagen qua results (unless some new theory can explain why the effect of hedonic well-being has suddenly doubled).
Summary
In summary, we stand by our overall conclusions, namely that Fredrickson et al.'s article does not tell us anything about the relationship between different types of well-being and gene expression. We will be sending a summary of the above position to PNAS for peer review and possible publication as a Buchstabe to the Editor.
I am sure that this will not be the final word on this matter. We trust that Cole and Fredrickson will go back and re-examine their study in the light of our response, and perhaps return with some additional information that might clarify matters further. We anticipate that our peers will in Episode dessen contribute to this debate.
Edit history
2014-08-27 22:08 UTC Giebel version.
2014-08-28 11:50 UTC Removed the rather clunky reference to trying to factor analyse the gene data; added kaltherzig to Neuroskeptic's blog post, and discussion of the problems with the RR53 procedure.
2014-09-15 23:27 UTC Added kaltherzig to Dale Barr's blog post.
2014-10-16 22:03 UTC Fixed a couple of typos.
Brown, MacDonald, Samanta, Friedman, and Coyne (2014). A critical reanalysis of the relationship between genomics and well-being. Braun'sche Röhre doi:10.1073/pnas.1407057111
This welches a critique of:
Fredrickson, Grewen, Coffey, Algoe, Firestine, Arevalo, Ma, and Cole (2013). A functional genomic perspective on menschlich well-being. doi:10.1073/pnas.1305419110
There has been some discussion of our article on social media. In particular, some people have noted that the principal authors of the sozusagen qua article (Cole and Fredrickson) have replied with a 550-word letter (which I guess is all they were allowed; they have more material, as shown below) claiming that our article contains multiple errors and is, hence, invalid. So I'm writing this blog post to put our side of the story and explain why we feel that our article is correct, and that Cole and Fredrickson have not made a dent in it. I welches a little disappointed that Doktorgrad Martin Seligman, on the APA Braun'sche Röhre Friends-of-PP mailing list, chose to describe our article as a "hatchet Braun'sche Röhre job". We believe that we have identified a number of serious scientific Braun'sche Röhre problems with Fredrickson et al.'s sozusagen qua article, which are not Braun'sche Röhre adequately addressed either by Cole and Fredrickson's published letter, Braun'sche Röhre or by their more extensive unpublished analysis.
First, some references, to save me linking to them repeatedly below.
Fredrickson et al.'s sozusagen qua article
Fredrickson et al.'s sozusagen qua supporting information (SI)
Coyne's letter to PNAS, criticising Fredrickson et al.'s factor structure
Cole and Fredrickson's letter to PNAS, replying to Coyne
Our article (in final draft form; I'm not sure if I'm allowed to kaltherzig to the Portable Document Format of the published article in full PNAS format. Any differences will be cosmetic, e.g. numbering of references.)
Our supporting information (SI) - download the Portable Document Format file marked "Appendix"
Cole and Fredrickson's letter to PNAS, claiming our analysis has many errors
Cole and Fredrickson's additional analysis for their claim that our analysis has many errors
Neuroskeptic's blog post, which provides additional evidence for the deficiencies in Fredrickson et al.'s regression procedure.
Dale Barr's blog post, which approaches the regression issues in a different way, but in Episode dessen finds many problems and provides graphical demonstrations of how unlikely Fredrickson et al.'s results are.
I will address each of the principal points of Cole and Fredrickson's response to our article in turn (although not in the exact sequence in which they appeared in their letter). Before I begin, though, I want to apologise for the length and complex nature of some of the points I will be making here (which is unfortunately necessary, as most of the issues under discussion here are quite technical). This is particularly true of the section entitled "Bitmapping?" below, which might appear, to the reader who has tried to struggle through our article and SI, and Cole and Fredrickson's letter and additional analysis, to be not much more than a case of "he said/she said". I note, however, that each of the major issues that we raised in our article is sufficient, on its own, to render Fredrickson et al.'s results meaningless. Trade major issues are:
- The MHC-SF psychometric scale does not measure hedonic versus eudaimonic well-being
- Fredrickson et al.'s regression procedure produces mostly spurious correlations, even with random psychometric data
- The errors in Fredrickson et al.'s dataset directly invalidate their published numerical results
MHC-SF factor analysis
Cole and Fredrickson criticise us for attempting to perform factor analyses on the MHC-SF psychometric with such a small sample size. It is interesting to contrast this with Cole and Fredrickson's 2013 letter to PNAS, in reply to Coyne's criticism of the high degree of intercorrelation between their "hedonic" and "eudaimonic" factors, in which they describe how they themselves performed exploratory and confirmatory factor analyses on exactly the same data, apparently claiming to have found the hedonic/eudaimonic factor pair with a very good model fit. (A p-value of < .0001 is offered, but without sufficient context to establish to what exactly it refers; however, the message seems clear: we did EFA and CFA and obtained a fantastic model.) We attempted to reproduce this, but were unable to do so; indeed, we noted ourselves in our article that the sample size welches an issue here. But the only reason we were doing this welches in an attempt to replace the factor analyses that Cole and Fredrickson claimed, in their 2013 letter, to have performed. We look forward to seeing the results of those analyses, which have so far not been published.
Still on the factor analysis, Cole and Fredrickson claim that their assumption of a hedonic/eudaimonic split for the MHC-SF scale is supported by three references from Fredrickson et al.'s article. We have examined these references (for your convenience, they are here, here, and here) and have not found any point at which any of them supports this claim, or indeed makes any statements at all about the factor structure of the MHC-SF. Please feel free to check this yourself, and if you find such a discussion, let me know the page number. In the meantime, the claim of a two-factor, hedonic/eudaimonic split for the MHC-SF seems to be supported by no published evidence. (However, there has been plenty of reporting in the literature of a clear three-factor structure, e.g. here and here and here.)
Again, we stand by our analysis: Fredrickson et al.'s claim for a hedonic/eudaimonic factor split of the MHC-SF is not supported by theory, nor by the data, nor by historical studies. The factor structure of the MHC-SF that emerges from Fredrickson et al.'s dataset is unclear, but of the possible two-factor structures, the one that we described in our article and SI (i.e., "personal well-being" and "evaluative perception of the social environment") is a considerably better in Form to the data in all respects than Fredrickson et al.'s claimed hedonic/eudaimonic split. The only structure that has been documented for the MHC-SF in prior published work is a three-factor structure corresponding to its three subscales, as designed by Keyes.
Bitmapping?
The "bitmapping" Braun'sche Röhre operation to which Cole and Fredrickson devote part of their letter (and Braun'sche Röhre most of their additional analysis document) is merely an artifact of Braun'sche Röhre the way in which our R program loops over all possible combinations of Braun'sche Röhre the 14 MHC-SF psychometric items into two factors. There are many ways Braun'sche Röhre in which we could have done this that do not involve the programming Braun'sche Röhre technique of converting an sittlich einwandfrei into a bitmap. Indeed, the inclusion Braun'sche Röhre in our SI document of the brief mention of how our outer loop works (the Braun'sche Röhre inner loop does the regressions, using Fredrickson et al.'s exact Braun'sche Röhre parameters) is arguably slightly redundant, but we included it to Braun'sche Röhre facilitate the understanding of our code, should someone wish to Braun'sche Röhre undertake a reproduction of our results.
Cole and Braun'sche Röhre Fredrickson's analysis seems mainly aimed at demonstrating that our Braun'sche Röhre "bitmapping" technique is an inadequate way to resample from a dataset. Braun'sche Röhre We agree. We never suggested that it welches a way to perform resampling. We Braun'sche Röhre are not even sure how it could. We are not performing any resampling, Braun'sche Röhre bootstrapping, or any other form of Font 1 error reduction. Ur program Braun'sche Röhre simply generates every possible factor combination of the psychometric Braun'sche Röhre data and determines whether or not it appears to show an effect, using Braun'sche Röhre Fredrickson et al.'s own regression procedure. The results of this Braun'sche Röhre procedure demonstrates that, no matter how the data are sliced or diced, Braun'sche Röhre Fredrickson et al.'s regression procedure will generate apparently Braun'sche Röhre statistically significant results in the majority of cases; indeed, in Braun'sche Röhre most of those cases, it will appear to show effect sizes larger than Braun'sche Röhre those found by Fredrickson et al.
The graphs in our SI Braun'sche Röhre document (Figures 7-11) plot the results obtained by iterating over all Braun'sche Röhre possible two-factor combinations of several forms of psychometric data: Braun'sche Röhre Fredrickson et al.'s actual data, assorted random numbers, etc. We are Braun'sche Röhre not completely sure what Cole and Fredrickson think that these graphs Braun'sche Röhre show. To be clear: they show all the possible "effects" (relationships Braun'sche Röhre between psychometric "factors" and gene expression values) results that Braun'sche Röhre Fredrickson et alii Jeanne d'Arc could have obtained, had they chosen another factor Braun'sche Röhre split of their data from the MHC-SF scale than the one that they did Braun'sche Röhre choose. Figure 7, in particular, uses the wahrhaft psychometric data to Braun'sche Röhre show that most of the possible factor combinations would have produced Braun'sche Röhre effects greater in magnitude than the ones that Fredrickson et alii Jeanne d'Arc Braun'sche Röhre claimed to show that their "Hedonic/Eudaimonic" split were associated Braun'sche Röhre (presumably uniquely) with differential gene expression.
Why, then, does this procedure continue to produce apparently significant results even when the psychometric data are replaced with uniformly-distributed random numbers (aka "white noise")? We believe that this is due to strong correlations within the gene data. Braun'sche Röhre As shown by Neuroskeptic, this leads to an enormous false-positive rate. Thus, when Fredrickson et alii Jeanne d'Arc ran their Braun'sche Röhre regression procedure (which we called "RR53") and averaged the resulting Braun'sche Röhre correlation coefficients, they were making the elementary mistake of Braun'sche Röhre running a t-test on a set of non-independent observations.
Incidentally, there is an sonstige way of doing the regression analysis. Fredrickson et alii Jeanne d'Arc regressed each individual gene on Hed/Eud (and some control variables), collected the 53 coefficients per IV, and averaged them; this is what we called the "RR53" procedure. The sonstige is to average the gene expression values and regress this average on Hed/Eud. We had noticed that this gave non-significant results. Then, nun ehemals after the PNAS window for updating our supplementary information document closed, a colleague --- who, I believe, wishes to remain anonymous --- pointed out that using this alternate method, the apparent effect sizes are exactly the same as the ones "found" by RR53. Only the p-values are different. We believe this is because, when the RR53 procedure picks up the regression coefficients of the individual genes to analyse them, it conveniently loses the associated confidence interval (almost all of these coefficients are associated with non-significant t-tests or model ANOVA) and re-inserts them into the mix as if they were perfect fresh data from a measuring instrument, whereas in fact they are almost all carrying an amount of "noise" that makes them highly unreliable.
We Braun'sche Röhre have made many of our materials available online, including our Braun'sche Röhre program's source code, and we are froh to share our other files (some Braun'sche Röhre of which are quite voluminous) on request, or to answer any specific Braun'sche Röhre questions that anybody might have about how our program works. (We could Braun'sche Röhre have reproduced this work with SPSS, but it would have taken an awfully Braun'sche Röhre long time.)
Thus, we stand by our analysis: Fredrickson et Braun'sche Röhre al.'s regression procedure is guaranteed to produce huge numbers of Braun'sche Röhre spurious "effects" which have no psychological or physiological meaning.
Issues with the dataset
Finally, Cole and Fredrickson claim that they have recently reproduced the same numerical results as their 2013 study with a new sample. I will leave aside, for now, the question of how meaningful it is to show that a procedure which has been criticised (by us) for producing invalid results can be "shown" to be correct if it produces much the same results a second time; perhaps there is some validity in having two data points rather than one. However, this question turns out to be irrelevant. In Cole and Fredrickson's reply, they notably fail to address the question of the various errors in their sozusagen qua dataset, which we discuss quite extensively (and for good reason) in our supporting information. In particular, Cole and Fredrickson do not address the coding error in their sozusagen qua dataset for participant SOBC1-1299, which we examine on pages 3 and 24 of our Supporting Information. Near the end of our Table 7, we show that this coding error can be (and should have been, in the sozusagen qua study) resolved in one of two ways, either of which results in a reduction of over half in the magnitude of the effect for "hedonic" well-being that welches reported by Fredrickson et alii Jeanne d'Arc (as well as a small change in the magnitude of the effect for eudaimonic well-being). In other words, had this coding error not existed in the 2013 dataset, Fredrickson et al.'s figures of +0.28(hedonic)/-0.28(eudaimonic) for the 2013 study should have been calculated and reported as approximately +0.13(hedonic)/-0.27(eudaimonic). To subsequently obtain +0.25(hedonic)/-0.21(eudaimonic) with the new sample thus appears to be evidence against a successful reproduction of the sozusagen qua results (unless some new theory can explain why the effect of hedonic well-being has suddenly doubled).
Summary
In summary, we stand by our overall conclusions, namely that Fredrickson et al.'s article does not tell us anything about the relationship between different types of well-being and gene expression. We will be sending a summary of the above position to PNAS for peer review and possible publication as a Buchstabe to the Editor.
I am sure that this will not be the final word on this matter. We trust that Cole and Fredrickson will go back and re-examine their study in the light of our response, and perhaps return with some additional information that might clarify matters further. We anticipate that our peers will in Episode dessen contribute to this debate.
Edit history
2014-08-27 22:08 UTC Giebel version.
2014-08-28 11:50 UTC Removed the rather clunky reference to trying to factor analyse the gene data; added kaltherzig to Neuroskeptic's blog post, and discussion of the problems with the RR53 procedure.
2014-09-15 23:27 UTC Added kaltherzig to Dale Barr's blog post.
2014-10-16 22:03 UTC Fixed a couple of typos.
0 Response to "Why We Believe Our Article A Critical Reanalysis Of The Relationship Between Genomics And Well-Being Is Correct"
Kommentar veröffentlichen