Methods of meta-analysis: correcting error and bias in research findings
Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic methods. Rather than taking an encyclopedic approach, the authors have focused on ca...
Gespeichert in:
| Späterer Titel: | Schmidt, Frank L., 1944- Methods of meta-analysis |
|---|---|
| Hauptverfasser: | , |
| Format: | Buch |
| Sprache: | English |
| Veröffentlicht: |
Thousand Oaks, Calif.
Sage
2004
|
| Ausgabe: | 2. ed. |
| Schlagworte: | |
| Online-Zugang: | Table of contents only Publisher description Inhaltsverzeichnis |
| Zusammenfassung: | Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic methods. Rather than taking an encyclopedic approach, the authors have focused on carefully developing those techniques that are most applicable to social science research, and have given a general conceptual description of more complex and rarely-used techniques. Fully revised and updated, the text includes new these new additions: an evaluation of fixed versus random effects models for meta-analysis; new methods for correcting for indirect range restriction in meta-analysis; new developments in corrections for measurement error; a discussion of a new Windows-based program package for applying the meta-analysis methods presented in the book; a presentation of the theories of data underlying different approaches to meta-analysis. |
| Beschreibung: | Hier auch später erschienene, unveränderte Nachdrucke |
| Beschreibung: | XXXIII, 582 S. graph. Darst. 27 cm |
| ISBN: | 1412909120 141290479X 9781412909129 9781412904797 |
Internformat
MARC
| LEADER | 00000nam a2200000zc 4500 | ||
|---|---|---|---|
| 001 | BV021819747 | ||
| 003 | DE-604 | ||
| 005 | 20221129 | ||
| 007 | t | ||
| 008 | 061121s2004 xxud||| |||| 00||| eng d | ||
| 010 | |a 2003026057 | ||
| 020 | |a 1412909120 |c cloth |9 1-412-90912-0 | ||
| 020 | |a 141290479X |c pbk. |9 1-412-90479-X | ||
| 020 | |a 9781412909129 |9 978-1-4129-0912-9 | ||
| 020 | |a 9781412904797 |9 978-1-4129-0479-7 | ||
| 035 | |a (OCoLC)53814539 | ||
| 035 | |a (DE-599)BVBBV021819747 | ||
| 040 | |a DE-604 |b ger |e aacr | ||
| 041 | 0 | |a eng | |
| 044 | |a xxu |c US | ||
| 049 | |a DE-N2 |a DE-19 |a DE-83 |a DE-11 |a DE-188 |a DE-945 |a DE-355 |a DE-523 |a DE-91 | ||
| 050 | 0 | |a HA29 | |
| 082 | 0 | |a 300/.72 | |
| 084 | |a CM 3000 |0 (DE-625)18945: |2 rvk | ||
| 084 | |a CM 4000 |0 (DE-625)18951: |2 rvk | ||
| 084 | |a PSY 470f |2 stub | ||
| 084 | |a SOZ 720f |2 stub | ||
| 084 | |a MAT 620f |2 stub | ||
| 084 | |a WIS 450f |2 stub | ||
| 100 | 1 | |a Hunter, John Edward |d 1939-2002 |e Verfasser |0 (DE-588)131404873 |4 aut | |
| 245 | 1 | 0 | |a Methods of meta-analysis |b correcting error and bias in research findings |c John E. Hunter ; Frank L. Schmidt |
| 250 | |a 2. ed. | ||
| 264 | 1 | |a Thousand Oaks, Calif. |b Sage |c 2004 | |
| 300 | |a XXXIII, 582 S. |b graph. Darst. |c 27 cm | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 500 | |a Hier auch später erschienene, unveränderte Nachdrucke | ||
| 520 | 3 | |a Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic methods. Rather than taking an encyclopedic approach, the authors have focused on carefully developing those techniques that are most applicable to social science research, and have given a general conceptual description of more complex and rarely-used techniques. Fully revised and updated, the text includes new these new additions: an evaluation of fixed versus random effects models for meta-analysis; new methods for correcting for indirect range restriction in meta-analysis; new developments in corrections for measurement error; a discussion of a new Windows-based program package for applying the meta-analysis methods presented in the book; a presentation of the theories of data underlying different approaches to meta-analysis. | |
| 650 | 4 | |a Ciencias sociales - Métodos estadísticos | |
| 650 | 7 | |a Meta-analyse |2 gtt | |
| 650 | 4 | |a Méta-analyse | |
| 650 | 4 | |a Sciences sociales - Méthodes statistiques | |
| 650 | 4 | |a Sozialwissenschaften | |
| 650 | 4 | |a Social sciences |x Statistical methods | |
| 650 | 4 | |a Meta-analysis | |
| 650 | 0 | 7 | |a Metaanalyse |0 (DE-588)4169552-5 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Methode |0 (DE-588)4038971-6 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Forschungsmethode |0 (DE-588)4155046-8 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Statistik |0 (DE-588)4056995-0 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Psychologie |0 (DE-588)4047704-6 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Metaanalyse |0 (DE-588)4169552-5 |D s |
| 689 | 0 | 1 | |a Forschungsmethode |0 (DE-588)4155046-8 |D s |
| 689 | 0 | |5 DE-604 | |
| 689 | 1 | 0 | |a Metaanalyse |0 (DE-588)4169552-5 |D s |
| 689 | 1 | 1 | |a Methode |0 (DE-588)4038971-6 |D s |
| 689 | 1 | 2 | |a Statistik |0 (DE-588)4056995-0 |D s |
| 689 | 1 | 3 | |a Psychologie |0 (DE-588)4047704-6 |D s |
| 689 | 1 | |8 1\p |5 DE-604 | |
| 700 | 1 | |a Schmidt, Frank L. |d 1944- |e Verfasser |0 (DE-588)131405470 |4 aut | |
| 785 | 0 | 0 | |i Gefolgt von |a Schmidt, Frank L., 1944- |t Methods of meta-analysis |
| 787 | 0 | 8 | |i Überarbeitet als |b 3. ed. |z 978-1-4522-8689-1 |w (DE-604)BV041790947 |
| 856 | 4 | |u http://www.loc.gov/catdir/toc/ecip0411/2003026057.html |3 Table of contents only | |
| 856 | 4 | |u http://www.loc.gov/catdir/enhancements/fy0658/2003026057-d.html |3 Publisher description | |
| 856 | 4 | 2 | |m HBZ Datenaustausch |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=015031909&sequence=000006&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-015031909 | ||
| 883 | 1 | |8 1\p |a cgwrk |d 20201028 |q DE-101 |u https://d-nb.info/provenance/plan#cgwrk | |
Datensatz im Suchindex
| _version_ | 1804135744618364928 |
|---|---|
| adam_text | Brief Contents
List of Tables xv
List of Figures xix
Preface to Second Edition xxiii
Preface to First Edition xxvii
Acknowledgments xxxiii
Part I: Introduction to Meta Analysis 1
1. Integrating Research Findings Across Studies 3
2. Study Artifacts and Their Impact on Study Outcomes 33
Part II: Meta Analysis of Correlations 73
3. Meta Analysis of Correlations Corrected Individually for Artifacts 75
4. Meta Analysis of Correlations Using Artifact Distributions 137
5. Technical Questions in Meta Analysis of Correlations 189
Part III: Meta Analysis of Experimental Effects
and Other Dichotomous Comparisons 241
6. Treatment Effects: Experimental Artifacts and Their Impact 243
7. Meta Analysis Methods for d Values 273
8. Technical Questions in Meta Analysis of d Values 335
Part IV: General Issues in Meta Analysis 391
9. General Technical Issues in Meta Analysis 393
10. Cumulation of Findings Within Studies 429
11. Methods of Integrating Findings Across
Studies and Related Software 445
12. Locating, Evaluating, Selecting, and Coding Studies 467
13. Availability and Source Bias in Meta Analysis 493
14. Summary of Psychometric Meta Analysis 511
Appendix 517
References 527
Name Index 563
Subject Index 569
About the Authors 581
Detailed Contents
List of Tables xv
List of Figures xix
Preface to Second Edition xxiii
Preface to First Edition xxvii
Acknowledgments xxxiii
Part I: Introduction to Meta Analysis 1
1: Integrating Research Findings Across Studies 3
General Problem and an Example 3
A Typical Interpretation of the Example Data 4
Conclusions of the Review 7
Critique of the Sample Review 7
Problems With Statistical Significance Tests 8
Is Statistical Power the Solution? 11
Confidence Intervals 13
Meta Analysis 15
Role of Meta Analysis in the Behavioral and Social Sciences 17
The Myth of the Perfect Study 17
Some Relevant History 18
Role of Meta Analysis in Theory Development 22
Increasing Use of Meta Analysis 24
Meta Analysis in Industrial Organizational Psychology 24
Wider Impact of Meta Analysis on Psychology 26
Impact of Meta Analysis Outside Psychology 28
Impact in Medicine 28
Impact in Other Disciplines 29
Meta Analysis and Social Policy 29
Meta Analysis and Theories of Data 30
Conclusions 32
2: Study Artifacts and Their Impact on Study Outcomes 33
Study Artifacts 34
Sampling Error 34
Error of Measurement 34
Dichotomization 36
Range Variation in the Independent Variable 37
Attrition Artifacts: Range Variation on the
Dependent Variable 39
Imperfect Construct Validity in the Independent Variable 41
Imperfect Construct Validity in the Dependent Variable 51
Computational and Other Errors in the Data 53
Extraneous Factors Introduced by Study Procedure 54
Bias in the Sample Correlation
Sampling Error, Statistical Power, and the Interpretation
of Research Findings 57
An Illustration of Statistical Power 57
A More Detailed Examination of Statistical Power 59
When and How to Cumulate 65
Undercorrection for Artifacts in the Corrected Standard Deviation 66
Coding Study Characteristics and Capitalization on Sampling
Error in Moderator Analysis 68
A Look Ahead in the Book 71
Part II: Meta Analysis of Correlations 73
3: Meta Analysis of Correlations Corrected
Individually for Artifacts 75
Introduction and Overview 75
Bare Bones Meta Analysis: Correcting for Sampling Error Only 81
Estimation of Sampling Error 81
Correcting the Variance for Sampling Error
and a Worked Example 83
Moderator Variables Analyzed by Grouping the Data
and a Worked Example 90
Correcting Feature Correlations for Sampling Error
and a Worked Example 2
Artifacts Other Than Sampling Error 95
Error of Measurement and Correction for Attenuation 95
Restriction or Enhancement of Range 103
Dichotomization of Independent and Dependent Variables 112
Imperfect Construct Validity in Independent and
Dependent Variables
Attrition Artifacts
Extraneous Factors
Bias in the Correlation
Multiple Simultaneous Artifacts
Meta Analysis of Individually Corrected Correlations 120
Individual Study Computations
Combining Across Studies
IOC
Final Meta Analysis Estimation
A Worked Example: Indirect Range Restriction 127
Summary of Meta Analysis Correcting Each Correlation
Individually 132
Exercise 1: Bare Bones Meta Analysis: Correcting for Sampling
Error Only 134
Exercise 2: Meta Analysis Correcting Each Correlation
Individually 135
4: Meta Analysis of Correlations Using Artifact Distributions 137
Full Artifact Distribution Meta Analysis 138
The Mean Correlation 140
The Standard Deviation of Correlations 142
A Worked Example: Error of Measurement 150
A Worked Example: Unreliability and
Direct Range Restriction 153
A Worked Example: Personnel Selection With Fixed Test
(Direct Range Restriction) 154
Personnel Selection With Varying Tests 158
Personnel Selection: Findings and Formulas in
the Literature 159
A Worked Example: Indirect Range Restriction
(Interactive Method) 166
Refinements to Increase Accuracy of the SD(, Estimate 168
Accuracy of Corrections for Artifacts 169
Mixed Meta Analysis: Partial Artifact Information in
Individual Studies 173
An Example: Dichotomization of Both Variables 175
Summary of Artifact Distribution Meta Analysis
of Correlations 180
Phase I: Cumulating Artifact Information 181
Phase 2a: Correcting the Mean Correlation 181
Phase 2b: Correcting the Standard Deviation of
Correlations 181
Exercise: Artifact Distribution Meta Analysis 183
5: Technical Questions in Meta Analysis of Correlations 189
;¦ Versus r2: Which Should Be Used . 189
/• Versus Regression Slopes and Intercepts
in Meta Analysis 192
Range Restriction 192
Measurement Error 192
Comparability of Units Across Studies 193
Comparability of Findings Across Meta Analyses 194
Intrinsic Interpretability 94
Technical Factors That Cause Overestimation of SDP 95
Presence of Non Pearson ts 195
Presence of Outliers and Other Data Errors 96
Use ofr Instead of r hi the Sampling Error Formula 197
Undercorrection for Sampling Error Variance in
the Presence of Range Restriction 198
Nonlinearity in the Range Correction 198
Other Factors Causing Overestimation o/SDp 200
Fixed and Random Effects Models in Meta Analysis 201
Accuracy of Different Random Effects Models 203
Credibility Versus Confidence Intervals in Meta Analysis 205
Computing Confidence Intervals in Meta Analysis 206
Range Restriction in Meta Analysis: New Technical Analysis 207
Domains With No Range Restriction 208
Random Measurement Error 209
Systematic Error of Measurement 210
Artificial Dichotomization 210
Multiple A rtifacts 211
Meta Analysis for Simple Artifacts 211
Direct Range Restriction 213
Range Restriction as a Single Artifact 213
Correction for Direct Range Restriction 215
Meta Analysis for Range Restriction as a Single Artifact 215
Two Populations in Direct Range Restriction 216
Error of Measurement in the Independent Variable in
Direct Range Restriction 216
Error of Measurement in the Dependent Variable in
Direct Range Restriction 219
Error of Measurement in Both Variables:
Direct Range Restriction 221
Meta Analysis in Direct Range Restriction: Previous Work 221
Educational and Employment Selection 222
Meta Analysis Correcting Correlations Individually:
Direct Range Restriction 223
Artifact Distribution Meta Analysis:
Direct Range Restriction 224
Indirect Range Restriction 224
A Causal Model for Indirect Range Restriction 226
Range Restriction on S 228
Range Restriction on Other Variables in Indirect
Range Restriction 228
Estimation in Indirect Range Restriction 229
The Correlation Between S and T in Indirect
Range Restriction 230
The Attenuation Model in Indirect Range Restriction 231
Predictor Measurement Error in Indirect Range Restriction 232
Meta Analysis Correcting Each Correlation Individually:
Indirect Range Restriction 233
Artifact Distribution Meta Analysis for Indirect
Range Restriction 233
Criticisms of Meta Analysis Procedures for Correlations 240
Part III: Meta Analysis of Experimental Effects and Other
Dichotomous Comparisons 241
6: Treatment Effects: Experimental
Artifacts and Their Impact 243
Quantification of the Treatment Effect: The d Statistic
and the Point Biserial Correlation 244
Sampling Error in d Values: Illustrations 247
Case 1: N = 30 248
Case 2: N = 68 250
Case 3: N = 400 251
Error of Measurement in the Dependent Variable 252
Error of Measurement in the Treatment Variable 256
Variation Across Studies in Treatment Strength 260
Range Variation on the Dependent Variable 261
Dichotomization of the Dependent Variable 262
Imperfect Construct Validity in the Dependent Variable 264
Imperfect Construct Validity in the Treatment Variable 266
Bias in the Effect Size (d Statistic) 266
Recording, Computational, and Transcriptional
Errors 268
Multiple Artifacts and Corrections 269
7: Meta Analysis Methods for d Values 273
Effect Size Indexes: d and r 275
Maximum Value of Point Biserial r 276
The Effect Size (A Statistic) 277
Correction of the Point Biserial
xfor Unequal Sample Sizes 279
Examples of the Convertibility ofr and d 280
Problems of Artificial Dichotomization 282
An Alternative to d: Glass s d 282
Sampling Error in the d Statistic 283
The Standard Error for d 283
The Confidence Interval for I) 286
Cumulation and Correction of the Variance
for Sampling Error 286
Bare Bones Meta Anahsis 287
A Worked Numerical Example 289
Another Example: Leadership Training by Experts 291
Analysis of Moderator Variables 292
Using Study Domain Subsets 293
Using Study Characteristic Correlations 294
A Worked Example: Training by Experts Versus Training
b Managers 295
Another Worked Example: Amount of Training 298
The Correlational Moderator A nalysis 301
Correcting d Value Statistics for Measurement Error in the
Dependent Variable 301
Meta Analysis ofd Values Corrected Individually and a
Worked Example 305
Artifact Distribution Meta Analysis and a Worked Example 308
Measurement Error in the Independent Variable in Experiments 313
Other Artifacts and Their Effects 315
Correcting for Multiple Artifacts 316
Attenuation Effect of Multiple Artifacts and
Correction for the Same 317
Disattenuation and Sampling Error:
The Confidence Interval 319
A Formula for Meta Analysis With Multiple Artifacts 320
Summary of Meta Analysis of d Values 328
Exercise: Meta Analysis of d Values 331
8: Technical Questions in Meta Analysis of d Values 335
Alternative Experimental Designs 335
Within Subjects Experimental Designs 337
The Potentially Perfect Power of the Pre Post Design 338
Deficiencies of the Between Subjects Design 339
Error of Measurement and the Within Subjects Design 344
The Treatment by Subjects Interaction 349
Sampling Error 356
Meta Analysis and the Within Subjects Design 370
The d Statistic 370
The Treatment by Subjects Interaction 371
Statistical Power in the Two Designs 374
Designs Matched for Number of Subjects 375
Designs Matched for Number of Measurements or Scores 378
Threats to Internal and External Validity 382
History 383
Maturation 384
Testing 384
Instrumentation 384
Regression to the Mean 385
Reactive Arrangements 386
Interaction Between Testing and Treatment 387
Interaction Between Selection and Treatment 387
Bias in Observed d Values 387
Use of Multiple Regression in Moderator Analysis of d Values 388
Part IV: General Issues in Meta Analysis 391
9: General Technical Issues in Meta Analysis 393
Fixed Effects Versus Random Effects Models in Meta Analysis 393
Second Order Sampling Error: General Principles 399
12: Locating, Evaluating, Selecting, and Coding Studies 467
Conducting a Thorough Literature Search 467
What to Do About Studies With Methodological Weaknesses 468
Coding Studies in Meta Analysis 470
What to Include in the Meta Analysis Report 471
Information Needed in Reports of Primary Studies 473
Correlational Studies 473
Experimental Studies 474
Studies Using Multiple Regression 475
Studies Using Factor Analysis 476
Studies Using Canonical Correlation 476
Studies Using Multivariate Analysis of Variance (MANOVA) All
General Comments on Reporting in Primary Studies All
Appendix: Coding Sheet for Validity Studies 479
13: Availability and Source Bias in Meta Analysis 493
Some Evidence on Publication Bias 494
Effects of Methodological Quality on Mean Effect Sizes
From Different Sources 495
Multiple Hypotheses and Other Considerations
in Availability Bias 496
Methods for Detecting Availability Bias 498
File Drawer Analysis Based on p Values 499
File Drawer Analysis Based on Effect Size 500
A Graphic Method for Detecting Availability Bias:
The Funnel Plot 501
Methods for Correcting for Availability Bias 503
The Original Hedges Olkin (1985) Method 504
The lyengar Greenhouse (1988) Method 505
The Begg Mazumdar (1994) Method 505
Further Work by Hedges and Associates 506
The Duval Tweedie (2000) Trim and Fill Method 508
Summary of Methods for Correcting Availability Bias 509
14: Summary of Psychometric Meta Analysis 511
Meta Analysis Methods and Theories of Data 511
What Is the Ultimate Purpose of Meta Analysis? 512
Psychometric Meta Analysis: Summary Overview 513
Appendix: Windows Based Meta Analysis Software Package 517
References 527
Name Index 563
Subject Index 569
About the Authors 581
List of Tables
Chapter 1
Table 1.1 Correlations between organizational commitment and
job satisfaction 4
Table 1.2 Existence of correlation between organizational
commitment and job satisfaction under various conditions
as shown by the studies in Table 1.1 5
Table 1.3 Analysis of correlations from studies based on younger
or mixed age subjects 6
Table 1.4 Analysis of correlations from studies based on younger
and mixed age subjects in small organizations 7
Table 1.5 Substantive meta analysis published in the top two journals
in industrial/organizational psychology by methods
used, 1974 2000 26
Chapter 2
Table 2.1 Study artifacts that alter the value of outcome measures
(with examples from personnel selection research) 35
Table 2.2 Nineteen studies 58
Table 2.3 Validity coefficients from the sears study 64
Table 2.4 Expected values of the multiple R of study characteristics
with study outcomes when all study characteristics
correlate zero with study outcomes and with each other 70
Chapter 3
Table 3.1 Study artifacts that alter the value of outcome measures
(with examples from personnel selection research) 76
Table 3.2 Correlations between socioeconomic status and police
brutality (U.S.) 90
Table 3.3 Correlations between socioeconomic status and police
brutality (U.S. and Transylvania) 92
Table 3.4 Tibetan employment service test validities 94
Detecting Moderators Not Hypothesized a Priori 401
Second Order Meta Analyses 406
Large N Studies and Meta Analysis 408
Second Order Sampling Error: Technical Treatment 411
The Homogeneous Case 415
The Heterogeneous Case 417
A Numerical Example 418
The Leadership Training by Experts Example 420
The Skills Training Moderator Example 421
The Detection of Moderator Variables: Summary 423
Hierarchical Analysis of Moderator Variables 424
Exercise: Second Order Meta Analysis 427
10: Cumulation of Findings Within Studies 429
Fully Replicated Designs 429
Conceptual Replication 430
Conceptual Replication and Confirmatory Factor Analysis 432
Conceptual Replication: An Alternative Approach 435
Analysis of Subgroups 439
Subgroups and Loss of Power 440
Subgroups and Capitalization on Chance 440
Subgroups and Suppression of Data 441
Subgroups and the Bias of Disaggregation 441
Conclusion: Use Total Group Correlations 442
Summary 442
11: Methods of Integrating Findings Across Studies and
Related Software 445
The Traditional Narrative Procedure 445
The Traditional Voting Method 446
Cumulation of p Values Across Studies 447
Statistically Correct Vote Counting Procedures 449
Vote Counting Methods Yielding Only
Significance Levels 449
Vote Counting Methods Yielding Estimates
of Effect Sizes 450
Meta Analysis of Research Studies 453
Descriptive Meta Analysis Methods: Glassian
and Related Methods 454
Meta Analysis Methods Focusing Only on Sampling Error:
Hedges s Methods, Rosenthal s Methods,
and Bare Bones Methods 45g
Psychometric Meta Analysis: Correction for
Multiple A rtifacts 46 ]
Unresolved Problems in Meta Analysis 463
Summary of Methods of Integrating Studies 463
Computer Programs for Meta Analysis 464
12: Locating, Evaluating, Selecting, and Coding Studies 467
Conducting a Thorough Literature Search 467
What to Do About Studies With Methodological Weaknesses 468
Coding Studies in Meta Analysis 470
What to Include in the Meta Analysis Report 471
Information Needed in Reports of Primary Studies 473
Correlational Studies 473
Experimental Studies 474
Studies Using Multiple Regression 475
Studies Using Factor Analysis 476
Studies Using Canonical Correlation 476
Studies Using Multivariate Analysis of Variance (MANOVA) 477
General Comments on Reporting in Primary Studies 477
Appendix: Coding Sheet for Validity Studies 479
13: Availability and Source Bias in Meta Analysis 493
Some Evidence on Publication Bias 494
Effects of Methodological Quality on Mean Effect Sizes
From Different Sources 495
Multiple Hypotheses and Other Considerations
in Availability Bias 496
Methods for Detecting Availability Bias 498
File Drawer Analysis Based on p Values 499
File Drawer Analysis Based on Effect Size 500
A Graphic Method for Detecting Availability Bias:
The Funnel Plot 501
Methods for Correcting for Availability Bias 503
The Original Hedges Olkin (1985) Method 504
The Iyengar Greenhouse (1988) Method 505
The Begg Mazumdar (1994) Method 505
Further Work by Hedges and Associates 506
The Duval Tweedie (2000) Trim and Fill Method 508
Summary of Methods for Correcting Availability Bias 509
14: Summary of Psychometric Meta Analysis 511
Meta Analysis Methods and Theories of Data 511
What Is the Ultimate Purpose of Meta Analysis? 512
Psychometric Meta Analysis: Summary Overview 513
Appendix: Windows Based Meta Analysis Software Package 517
References 527
Name Index 563
Subject Index 569
About the Authors 581
List of Tables
Chapter 1
Table 1.1 Correlations between organizational commitment and
job satisfaction 4
Table 1.2 Existence of correlation between organizational
commitment and job satisfaction under various conditions
as shown by the studies in Table 1.1 5
Table 1.3 Analysis of correlations from studies based on younger
or mixed age subjects 6
Table 1.4 Analysis of correlations from studies based on younger
and mixed age subjects in small organizations 7
Table 1.5 Substantive meta analysis published in the top two journals
in industrial/organizational psychology by methods
used, 1974 2000 26
Chapter 2
Table 2.1 Study artifacts that alter the value of outcome measures
(with examples from personnel selection research) 35
Table 2.2 Nineteen studies 58
Table 2.3 Validity coefficients from the sears study 64
Table 2.4 Expected values of the multiple R of study characteristics
with study outcomes when all study characteristics
correlate zero with study outcomes and with each other 70
Chapter 3
Table 3.1 Study artifacts that alter the value of outcome measures
(with examples from personnel selection research) 76
Table 3.2 Correlations between socioeconomic status and police
brutality (U.S.) 90
Table 3.3 Correlations between socioeconomic status and police
brutality (U.S. and Transylvania) 92
Table 3.4 Tibetan employment service test validities 94
Table 3.5 A meta analysis on hypothetical personnel selection
studies with indirect range restriction 129
Table 3.5a Hypothetical validity and artifact information 129
Table 3.5b Meta analysis worksheet 129
Chapter 4
Table 4.1 Organizational commitment and job satisfaction
(hypothetical results) 151
Table 4.1a Basic information 151
Table 4.1b Meta analysis worksheet 151
Table 4.2 Sixteen hypothetical studies 153
Table 4.3 Meta analysis of personnel selection validities
(direct range restriction) 156
Table 4.4 Indirect range restriction: Hypothetical data for
an artifact distribution meta analysis of 12 personnel
selection studies 166
Table 4.5 Hypothetical meta analysis of performance and turnover 176
Table 4.5a Basic study information 176
Table 4.5b Attenuation factors 177
Table 4.5c Worksheet for the interim meta analysis 178
Chapter 5—No Tables
Chapter 6
Table 6.1 Hypothetical meta analysis data for the effect of language
intensity on persuasion (results ordered by magnitude) 248
Table 6.2 Power of the conventional significance test for studies
with sample size N = 100 256
Table 6.3 Comparison ratio of corrected/actual
correlations—expressed as percentages—where the
corrected correlation is the estimated correlation for the
continuous dependent variable computed by correcting
the dichotomous variable correlation using the biserial
correction formula 264
Chapter 7
Table 7.1 Leadership training (studies with training by
outside experts) 292
Table 7.2 Training in interpersonal skills by managers versus
by experts 296
Table 7.3 Training in interpersonal skills by hours 299
Table 7.4 Bare bones meta analyses on hypothetical studies on
training doppelgangers 306
Table 7.5 Worksheet for meta analysis of studies in Table 7.4 308
Table 7.6 Meta analysis of hypothetical studies examining the
effect of a failure experience on state anxiety 311
Table 7.7 Meta analyses performed on the studies of failure and
anxiety in Table 7.6 312
Chapter 8
Table 8.1 Power of the within subjects and independent groups
designs for two effect sizes when the number of subjects
is the same in both designs 377
Table 8.2 Power of the within subjects and independent groups
designs for two effect sizes when the number of
measurements is the same in both designs 381
Chapter 9
Table 9.1 Second order sampling error: schematic showing when
the two types of second order sampling error occur 412
Chapter 10—No Tables
Chapter 11—No Tables
Chapter 12—No Tables
Chapter 13
Table 13.1 Hypothetical example of observed and true mean effect
sizes for four sources of effect sizes 496
Chapter 14—No Tables
Appendix—No Tables
Figure 2.7a Path diagram for an extraneous variable Z introduced
by study procedure 55
Figure 2.7b Experience differences as an extraneous variable
produced by concurrent validation procedure 55
Figure 2.8 Statistical power: Two examples 61
Figure 2.8a Statistical power greater than .50 61
Figure 2.8b Statistical power less than .50 61
Chapter 3—No Figures
Chapter 4—No Figures
Chapter 5
Figure 5.1 Path diagram for the applicant population in indirect range 227
Chapter 6
Figure 6.1 Effect of error of measurement on the separation
between the control and experimental groups for a case
in which the true score treatment effect is 8 = 1.00 255
Figure 6.1a Perfect measurement, ryy = 1.00 255
Figure 6.1b Typical good measurement, ryy = 81 255
Figure 6.1c Typical moderate measurement, ryy = .50 255
Figure 6.Id Typical poor measurement, ryy = .25 255
Figure 6.2 Path model for the relationship among the delinquency
treatment program, the desired measure of actual
posttreatment behavior, and the observed posttreatment
arrest record 265
Figure 6.3 Path model for the assumed relationship among the
social skills training of supervisors, program mastery,
and subsequent interpersonal behavior on the job 266
Chapter 7—No Figures
Chapter 8—No Figures
Chapter 9—No Figures
Chapter 10—No Figures
Chapter 11
Figure 11.1 Schematic illustrating methods of meta analysis 455
Chapter 12—No Figures
Chapter 13
Figure 13.1 Example of funnel plot showing no evidence of
availability bias 502
Figure 13.2 Example of funnel plot showing evidence of
availability bias 502
Chapter 14—No Figures
Appendix—No Figures
|
| adam_txt |
Brief Contents
List of Tables xv
List of Figures xix
Preface to Second Edition xxiii
Preface to First Edition xxvii
Acknowledgments xxxiii
Part I: Introduction to Meta Analysis 1
1. Integrating Research Findings Across Studies 3
2. Study Artifacts and Their Impact on Study Outcomes 33
Part II: Meta Analysis of Correlations 73
3. Meta Analysis of Correlations Corrected Individually for Artifacts 75
4. Meta Analysis of Correlations Using Artifact Distributions 137
5. Technical Questions in Meta Analysis of Correlations 189
Part III: Meta Analysis of Experimental Effects
and Other Dichotomous Comparisons 241
6. Treatment Effects: Experimental Artifacts and Their Impact 243
7. Meta Analysis Methods for d Values 273
8. Technical Questions in Meta Analysis of d Values 335
Part IV: General Issues in Meta Analysis 391
9. General Technical Issues in Meta Analysis 393
10. Cumulation of Findings Within Studies 429
11. Methods of Integrating Findings Across
Studies and Related Software 445
12. Locating, Evaluating, Selecting, and Coding Studies 467
13. Availability and Source Bias in Meta Analysis 493
14. Summary of Psychometric Meta Analysis 511
Appendix 517
References 527
Name Index 563
Subject Index 569
About the Authors 581
Detailed Contents
List of Tables xv
List of Figures xix
Preface to Second Edition xxiii
Preface to First Edition xxvii
Acknowledgments xxxiii
Part I: Introduction to Meta Analysis 1
1: Integrating Research Findings Across Studies 3
General Problem and an Example 3
A Typical Interpretation of the Example Data 4
Conclusions of the Review 7
Critique of the Sample Review 7
Problems With Statistical Significance Tests 8
Is Statistical Power the Solution? 11
Confidence Intervals 13
Meta Analysis 15
Role of Meta Analysis in the Behavioral and Social Sciences 17
The Myth of the Perfect Study 17
Some Relevant History 18
Role of Meta Analysis in Theory Development 22
Increasing Use of Meta Analysis 24
Meta Analysis in Industrial Organizational Psychology 24
Wider Impact of Meta Analysis on Psychology 26
Impact of Meta Analysis Outside Psychology 28
Impact in Medicine 28
Impact in Other Disciplines 29
Meta Analysis and Social Policy 29
Meta Analysis and Theories of Data 30
Conclusions 32
2: Study Artifacts and Their Impact on Study Outcomes 33
Study Artifacts 34
Sampling Error 34
Error of Measurement 34
Dichotomization 36
Range Variation in the Independent Variable 37
Attrition Artifacts: Range Variation on the
Dependent Variable 39
Imperfect Construct Validity in the Independent Variable 41
Imperfect Construct Validity in the Dependent Variable 51
Computational and Other Errors in the Data 53
Extraneous Factors Introduced by Study Procedure 54
Bias in the Sample Correlation "
Sampling Error, Statistical Power, and the Interpretation
of Research Findings 57
An Illustration of Statistical Power 57
A More Detailed Examination of Statistical Power 59
When and How to Cumulate 65
Undercorrection for Artifacts in the Corrected Standard Deviation 66
Coding Study Characteristics and Capitalization on Sampling
Error in Moderator Analysis 68
A Look Ahead in the Book 71
Part II: Meta Analysis of Correlations 73
3: Meta Analysis of Correlations Corrected
Individually for Artifacts 75
Introduction and Overview 75
Bare Bones Meta Analysis: Correcting for Sampling Error Only 81
Estimation of Sampling Error 81
Correcting the Variance for Sampling Error
and a Worked Example 83
Moderator Variables Analyzed by Grouping the Data
and a Worked Example 90
Correcting Feature Correlations for Sampling Error
and a Worked Example "2
Artifacts Other Than Sampling Error 95
Error of Measurement and Correction for Attenuation 95
Restriction or Enhancement of Range 103
Dichotomization of Independent and Dependent Variables 112
Imperfect Construct Validity in Independent and
Dependent Variables
Attrition Artifacts
Extraneous Factors
Bias in the Correlation
Multiple Simultaneous Artifacts
Meta Analysis of Individually Corrected Correlations 120
Individual Study Computations
Combining Across Studies
IOC
Final Meta Analysis Estimation
A Worked Example: Indirect Range Restriction 127
Summary of Meta Analysis Correcting Each Correlation
Individually 132
Exercise 1: Bare Bones Meta Analysis: Correcting for Sampling
Error Only 134
Exercise 2: Meta Analysis Correcting Each Correlation
Individually 135
4: Meta Analysis of Correlations Using Artifact Distributions 137
Full Artifact Distribution Meta Analysis 138
The Mean Correlation 140
The Standard Deviation of Correlations 142
A Worked Example: Error of Measurement 150
A Worked Example: Unreliability and
Direct Range Restriction 153
A Worked Example: Personnel Selection With Fixed Test
(Direct Range Restriction) 154
Personnel Selection With Varying Tests 158
Personnel Selection: Findings and Formulas in
the Literature 159
A Worked Example: Indirect Range Restriction
(Interactive Method) 166
Refinements to Increase Accuracy of the SD(, Estimate 168
Accuracy of Corrections for Artifacts 169
Mixed Meta Analysis: Partial Artifact Information in
Individual Studies 173
An Example: Dichotomization of Both Variables 175
Summary of Artifact Distribution Meta Analysis
of Correlations 180
Phase I: Cumulating Artifact Information 181
Phase 2a: Correcting the Mean Correlation 181
Phase 2b: Correcting the Standard Deviation of
Correlations 181
Exercise: Artifact Distribution Meta Analysis 183
5: Technical Questions in Meta Analysis of Correlations 189
;¦ Versus r2: Which Should Be Used'.' 189
/• Versus Regression Slopes and Intercepts
in Meta Analysis 192
Range Restriction 192
Measurement Error 192
Comparability of Units Across Studies 193
Comparability of Findings Across Meta Analyses 194
Intrinsic Interpretability \ 94
Technical Factors That Cause Overestimation of SDP \ 95
Presence of Non Pearson ts 195
Presence of Outliers and Other Data Errors \ 96
Use ofr Instead of r hi the Sampling Error Formula 197
Undercorrection for Sampling Error Variance in
the Presence of Range Restriction 198
Nonlinearity in the Range Correction 198
Other Factors Causing Overestimation o/SDp 200
Fixed and Random Effects Models in Meta Analysis 201
Accuracy of Different Random Effects Models 203
Credibility Versus Confidence Intervals in Meta Analysis 205
Computing Confidence Intervals in Meta Analysis 206
Range Restriction in Meta Analysis: New Technical Analysis 207
Domains With No Range Restriction 208
Random Measurement Error 209
Systematic Error of Measurement 210
Artificial Dichotomization 210
Multiple A rtifacts 211
Meta Analysis for Simple Artifacts 211
Direct Range Restriction 213
Range Restriction as a Single Artifact 213
Correction for Direct Range Restriction 215
Meta Analysis for Range Restriction as a Single Artifact 215
Two Populations in Direct Range Restriction 216
Error of Measurement in the Independent Variable in
Direct Range Restriction 216
Error of Measurement in the Dependent Variable in
Direct Range Restriction 219
Error of Measurement in Both Variables:
Direct Range Restriction 221
Meta Analysis in Direct Range Restriction: Previous Work 221
Educational and Employment Selection 222
Meta Analysis Correcting Correlations Individually:
Direct Range Restriction 223
Artifact Distribution Meta Analysis:
Direct Range Restriction 224
Indirect Range Restriction 224
A Causal Model for Indirect Range Restriction 226
Range Restriction on S 228
Range Restriction on Other Variables in Indirect
Range Restriction 228
Estimation in Indirect Range Restriction 229
The Correlation Between S and T in Indirect
Range Restriction 230
The Attenuation Model in Indirect Range Restriction 231
Predictor Measurement Error in Indirect Range Restriction 232
Meta Analysis Correcting Each Correlation Individually:
Indirect Range Restriction 233
Artifact Distribution Meta Analysis for Indirect
Range Restriction 233
Criticisms of Meta Analysis Procedures for Correlations 240
Part III: Meta Analysis of Experimental Effects and Other
Dichotomous Comparisons 241
6: Treatment Effects: Experimental
Artifacts and Their Impact 243
Quantification of the Treatment Effect: The d Statistic
and the Point Biserial Correlation 244
Sampling Error in d Values: Illustrations 247
Case 1: N = 30 248
Case 2: N = 68 250
Case 3: N = 400 251
Error of Measurement in the Dependent Variable 252
Error of Measurement in the Treatment Variable 256
Variation Across Studies in Treatment Strength 260
Range Variation on the Dependent Variable 261
Dichotomization of the Dependent Variable 262
Imperfect Construct Validity in the Dependent Variable 264
Imperfect Construct Validity in the Treatment Variable 266
Bias in the Effect Size (d Statistic) 266
Recording, Computational, and Transcriptional
Errors 268
Multiple Artifacts and Corrections 269
7: Meta Analysis Methods for d Values 273
Effect Size Indexes: d and r 275
Maximum Value of Point Biserial r 276
The Effect Size (A Statistic) 277
Correction of the Point Biserial
xfor Unequal Sample Sizes 279
Examples of the Convertibility ofr and d 280
Problems of Artificial Dichotomization 282
An Alternative to d: Glass's d 282
Sampling Error in the d Statistic 283
The Standard Error for d 283
The Confidence Interval for I) 286
Cumulation and Correction of the Variance
for Sampling Error 286
Bare Bones Meta Anahsis 287
A Worked Numerical Example 289
Another Example: Leadership Training by Experts 291
Analysis of Moderator Variables 292
Using Study Domain Subsets 293
Using Study Characteristic Correlations 294
A Worked Example: Training by Experts Versus Training
b\ Managers 295
Another Worked Example: Amount of Training 298
The Correlational Moderator A nalysis 301
Correcting d Value Statistics for Measurement Error in the
Dependent Variable 301
Meta Analysis ofd Values Corrected Individually and a
Worked Example 305
Artifact Distribution Meta Analysis and a Worked Example 308
Measurement Error in the Independent Variable in Experiments 313
Other Artifacts and Their Effects 315
Correcting for Multiple Artifacts 316
Attenuation Effect of Multiple Artifacts and
Correction for the Same 317
Disattenuation and Sampling Error:
The Confidence Interval 319
A Formula for Meta Analysis With Multiple Artifacts 320
Summary of Meta Analysis of d Values 328
Exercise: Meta Analysis of d Values 331
8: Technical Questions in Meta Analysis of d Values 335
Alternative Experimental Designs 335
Within Subjects Experimental Designs 337
The Potentially Perfect Power of the Pre Post Design 338
Deficiencies of the Between Subjects Design 339
Error of Measurement and the Within Subjects Design 344
The Treatment by Subjects Interaction 349
Sampling Error 356
Meta Analysis and the Within Subjects Design 370
The d Statistic 370
The Treatment by Subjects Interaction 371
Statistical Power in the Two Designs 374
Designs Matched for Number of Subjects 375
Designs Matched for Number of Measurements or Scores 378
Threats to Internal and External Validity 382
History 383
Maturation 384
Testing 384
Instrumentation 384
Regression to the Mean 385
Reactive Arrangements 386
Interaction Between Testing and Treatment 387
Interaction Between Selection and Treatment 387
Bias in Observed d Values 387
Use of Multiple Regression in Moderator Analysis of d Values 388
Part IV: General Issues in Meta Analysis 391
9: General Technical Issues in Meta Analysis 393
Fixed Effects Versus Random Effects Models in Meta Analysis 393
Second Order Sampling Error: General Principles 399
12: Locating, Evaluating, Selecting, and Coding Studies 467
Conducting a Thorough Literature Search 467
What to Do About Studies With Methodological Weaknesses 468
Coding Studies in Meta Analysis 470
What to Include in the Meta Analysis Report 471
Information Needed in Reports of Primary Studies 473
Correlational Studies 473
Experimental Studies 474
Studies Using Multiple Regression 475
Studies Using Factor Analysis 476
Studies Using Canonical Correlation 476
Studies Using Multivariate Analysis of Variance (MANOVA) All
General Comments on Reporting in Primary Studies All
Appendix: Coding Sheet for Validity Studies 479
13: Availability and Source Bias in Meta Analysis 493
Some Evidence on Publication Bias 494
Effects of Methodological Quality on Mean Effect Sizes
From Different Sources 495
Multiple Hypotheses and Other Considerations
in Availability Bias 496
Methods for Detecting Availability Bias 498
File Drawer Analysis Based on p Values 499
File Drawer Analysis Based on Effect Size 500
A Graphic Method for Detecting Availability Bias:
The Funnel Plot 501
Methods for Correcting for Availability Bias 503
The Original Hedges Olkin (1985) Method 504
The lyengar Greenhouse (1988) Method 505
The Begg Mazumdar (1994) Method 505
Further Work by Hedges and Associates 506
The Duval Tweedie (2000) Trim and Fill Method 508
Summary of Methods for Correcting Availability Bias 509
14: Summary of Psychometric Meta Analysis 511
Meta Analysis Methods and Theories of Data 511
What Is the Ultimate Purpose of Meta Analysis? 512
Psychometric Meta Analysis: Summary Overview 513
Appendix: Windows Based Meta Analysis Software Package 517
References 527
Name Index 563
Subject Index 569
About the Authors 581
List of Tables
Chapter 1
Table 1.1 Correlations between organizational commitment and
job satisfaction 4
Table 1.2 Existence of correlation between organizational
commitment and job satisfaction under various conditions
as shown by the studies in Table 1.1 5
Table 1.3 Analysis of correlations from studies based on younger
or mixed age subjects 6
Table 1.4 Analysis of correlations from studies based on younger
and mixed age subjects in small organizations 7
Table 1.5 Substantive meta analysis published in the top two journals
in industrial/organizational psychology by methods
used, 1974 2000 26
Chapter 2
Table 2.1 Study artifacts that alter the value of outcome measures
(with examples from personnel selection research) 35
Table 2.2 Nineteen studies 58
Table 2.3 Validity coefficients from the sears study 64
Table 2.4 Expected values of the multiple R of study characteristics
with study outcomes when all study characteristics
correlate zero with study outcomes and with each other 70
Chapter 3
Table 3.1 Study artifacts that alter the value of outcome measures
(with examples from personnel selection research) 76
Table 3.2 Correlations between socioeconomic status and police
brutality (U.S.) 90
Table 3.3 Correlations between socioeconomic status and police
brutality (U.S. and Transylvania) 92
Table 3.4 Tibetan employment service test validities 94
Detecting Moderators Not Hypothesized a Priori 401
Second Order Meta Analyses 406
Large N Studies and Meta Analysis 408
Second Order Sampling Error: Technical Treatment 411
The Homogeneous Case 415
The Heterogeneous Case 417
A Numerical Example 418
The Leadership Training by Experts Example 420
The Skills Training Moderator Example 421
The Detection of Moderator Variables: Summary 423
Hierarchical Analysis of Moderator Variables 424
Exercise: Second Order Meta Analysis 427
10: Cumulation of Findings Within Studies 429
Fully Replicated Designs 429
Conceptual Replication 430
Conceptual Replication and Confirmatory Factor Analysis 432
Conceptual Replication: An Alternative Approach 435
Analysis of Subgroups 439
Subgroups and Loss of Power 440
Subgroups and Capitalization on Chance 440
Subgroups and Suppression of Data 441
Subgroups and the Bias of Disaggregation 441
Conclusion: Use Total Group Correlations 442
Summary 442
11: Methods of Integrating Findings Across Studies and
Related Software 445
The Traditional Narrative Procedure 445
The Traditional Voting Method 446
Cumulation of p Values Across Studies 447
Statistically Correct Vote Counting Procedures 449
Vote Counting Methods Yielding Only
Significance Levels 449
Vote Counting Methods Yielding Estimates
of Effect Sizes 450
Meta Analysis of Research Studies 453
Descriptive Meta Analysis Methods: Glassian
and Related Methods 454
Meta Analysis Methods Focusing Only on Sampling Error:
Hedges's Methods, Rosenthal 's Methods,
and Bare Bones Methods 45g
Psychometric Meta Analysis: Correction for
Multiple A rtifacts 46 ]
Unresolved Problems in Meta Analysis 463
Summary of Methods of Integrating Studies 463
Computer Programs for Meta Analysis 464
12: Locating, Evaluating, Selecting, and Coding Studies 467
Conducting a Thorough Literature Search 467
What to Do About Studies With Methodological Weaknesses 468
Coding Studies in Meta Analysis 470
What to Include in the Meta Analysis Report 471
Information Needed in Reports of Primary Studies 473
Correlational Studies 473
Experimental Studies 474
Studies Using Multiple Regression 475
Studies Using Factor Analysis 476
Studies Using Canonical Correlation 476
Studies Using Multivariate Analysis of Variance (MANOVA) 477
General Comments on Reporting in Primary Studies 477
Appendix: Coding Sheet for Validity Studies 479
13: Availability and Source Bias in Meta Analysis 493
Some Evidence on Publication Bias 494
Effects of Methodological Quality on Mean Effect Sizes
From Different Sources 495
Multiple Hypotheses and Other Considerations
in Availability Bias 496
Methods for Detecting Availability Bias 498
File Drawer Analysis Based on p Values 499
File Drawer Analysis Based on Effect Size 500
A Graphic Method for Detecting Availability Bias:
The Funnel Plot 501
Methods for Correcting for Availability Bias 503
The Original Hedges Olkin (1985) Method 504
The Iyengar Greenhouse (1988) Method 505
The Begg Mazumdar (1994) Method 505
Further Work by Hedges and Associates 506
The Duval Tweedie (2000) Trim and Fill Method 508
Summary of Methods for Correcting Availability Bias 509
14: Summary of Psychometric Meta Analysis 511
Meta Analysis Methods and Theories of Data 511
What Is the Ultimate Purpose of Meta Analysis? 512
Psychometric Meta Analysis: Summary Overview 513
Appendix: Windows Based Meta Analysis Software Package 517
References 527
Name Index 563
Subject Index 569
About the Authors 581
List of Tables
Chapter 1
Table 1.1 Correlations between organizational commitment and
job satisfaction 4
Table 1.2 Existence of correlation between organizational
commitment and job satisfaction under various conditions
as shown by the studies in Table 1.1 5
Table 1.3 Analysis of correlations from studies based on younger
or mixed age subjects 6
Table 1.4 Analysis of correlations from studies based on younger
and mixed age subjects in small organizations 7
Table 1.5 Substantive meta analysis published in the top two journals
in industrial/organizational psychology by methods
used, 1974 2000 26
Chapter 2
Table 2.1 Study artifacts that alter the value of outcome measures
(with examples from personnel selection research) 35
Table 2.2 Nineteen studies 58
Table 2.3 Validity coefficients from the sears study 64
Table 2.4 Expected values of the multiple R of study characteristics
with study outcomes when all study characteristics
correlate zero with study outcomes and with each other 70
Chapter 3
Table 3.1 Study artifacts that alter the value of outcome measures
(with examples from personnel selection research) 76
Table 3.2 Correlations between socioeconomic status and police
brutality (U.S.) 90
Table 3.3 Correlations between socioeconomic status and police
brutality (U.S. and Transylvania) 92
Table 3.4 Tibetan employment service test validities 94
Table 3.5 A meta analysis on hypothetical personnel selection
studies with indirect range restriction 129
Table 3.5a Hypothetical validity and artifact information 129
Table 3.5b Meta analysis worksheet 129
Chapter 4
Table 4.1 Organizational commitment and job satisfaction
(hypothetical results) 151
Table 4.1a Basic information 151
Table 4.1b Meta analysis worksheet 151
Table 4.2 Sixteen hypothetical studies 153
Table 4.3 Meta analysis of personnel selection validities
(direct range restriction) 156
Table 4.4 Indirect range restriction: Hypothetical data for
an artifact distribution meta analysis of 12 personnel
selection studies 166
Table 4.5 Hypothetical meta analysis of performance and turnover 176
Table 4.5a Basic study information 176
Table 4.5b Attenuation factors 177
Table 4.5c Worksheet for the interim meta analysis 178
Chapter 5—No Tables
Chapter 6
Table 6.1 Hypothetical meta analysis data for the effect of language
intensity on persuasion (results ordered by magnitude) 248
Table 6.2 Power of the conventional significance test for studies
with sample size N = 100 256
Table 6.3 Comparison ratio of corrected/actual
correlations—expressed as percentages—where the
corrected correlation is the estimated correlation for the
continuous dependent variable computed by correcting
the dichotomous variable correlation using the biserial
correction formula 264
Chapter 7
Table 7.1 Leadership training (studies with training by
outside experts) 292
Table 7.2 Training in interpersonal skills by managers versus
by experts 296
Table 7.3 Training in interpersonal skills by hours 299
Table 7.4 Bare bones meta analyses on hypothetical studies on
training doppelgangers 306
Table 7.5 Worksheet for meta analysis of studies in Table 7.4 308
Table 7.6 Meta analysis of hypothetical studies examining the
effect of a failure experience on state anxiety 311
Table 7.7 Meta analyses performed on the studies of failure and
anxiety in Table 7.6 312
Chapter 8
Table 8.1 Power of the within subjects and independent groups
designs for two effect sizes when the number of subjects
is the same in both designs 377
Table 8.2 Power of the within subjects and independent groups
designs for two effect sizes when the number of
measurements is the same in both designs 381
Chapter 9
Table 9.1 Second order sampling error: schematic showing when
the two types of second order sampling error occur 412
Chapter 10—No Tables
Chapter 11—No Tables
Chapter 12—No Tables
Chapter 13
Table 13.1 Hypothetical example of observed and true mean effect
sizes for four sources of effect sizes 496
Chapter 14—No Tables
Appendix—No Tables
Figure 2.7a Path diagram for an extraneous variable Z introduced
by study procedure 55
Figure 2.7b Experience differences as an extraneous variable
produced by concurrent validation procedure 55
Figure 2.8 Statistical power: Two examples 61
Figure 2.8a Statistical power greater than .50 61
Figure 2.8b Statistical power less than .50 61
Chapter 3—No Figures
Chapter 4—No Figures
Chapter 5
Figure 5.1 Path diagram for the applicant population in indirect range 227
Chapter 6
Figure 6.1 Effect of error of measurement on the separation
between the control and experimental groups for a case
in which the true score treatment effect is 8 = 1.00 255
Figure 6.1a Perfect measurement, ryy = 1.00 255
Figure 6.1b Typical good measurement, ryy = 81 255
Figure 6.1c Typical moderate measurement, ryy = .50 255
Figure 6.Id Typical poor measurement, ryy = .25 255
Figure 6.2 Path model for the relationship among the delinquency
treatment program, the desired measure of actual
posttreatment behavior, and the observed posttreatment
arrest record 265
Figure 6.3 Path model for the assumed relationship among the
social skills training of supervisors, program mastery,
and subsequent interpersonal behavior on the job 266
Chapter 7—No Figures
Chapter 8—No Figures
Chapter 9—No Figures
Chapter 10—No Figures
Chapter 11
Figure 11.1 Schematic illustrating methods of meta analysis 455
Chapter 12—No Figures
Chapter 13
Figure 13.1 Example of funnel plot showing no evidence of
availability bias 502
Figure 13.2 Example of funnel plot showing evidence of
availability bias 502
Chapter 14—No Figures
Appendix—No Figures |
| any_adam_object | 1 |
| any_adam_object_boolean | 1 |
| author | Hunter, John Edward 1939-2002 Schmidt, Frank L. 1944- |
| author_GND | (DE-588)131404873 (DE-588)131405470 |
| author_facet | Hunter, John Edward 1939-2002 Schmidt, Frank L. 1944- |
| author_role | aut aut |
| author_sort | Hunter, John Edward 1939-2002 |
| author_variant | j e h je jeh f l s fl fls |
| building | Verbundindex |
| bvnumber | BV021819747 |
| callnumber-first | H - Social Science |
| callnumber-label | HA29 |
| callnumber-raw | HA29 |
| callnumber-search | HA29 |
| callnumber-sort | HA 229 |
| callnumber-subject | HA - Statistics |
| classification_rvk | CM 3000 CM 4000 |
| classification_tum | PSY 470f SOZ 720f MAT 620f WIS 450f |
| ctrlnum | (OCoLC)53814539 (DE-599)BVBBV021819747 |
| dewey-full | 300/.72 |
| dewey-hundreds | 300 - Social sciences |
| dewey-ones | 300 - Social sciences |
| dewey-raw | 300/.72 |
| dewey-search | 300/.72 |
| dewey-sort | 3300 272 |
| dewey-tens | 300 - Social sciences |
| discipline | Soziologie Psychologie Mathematik Wissenschaftskunde |
| discipline_str_mv | Soziologie Psychologie Mathematik Wissenschaftskunde |
| edition | 2. ed. |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>04127nam a2200769zc 4500</leader><controlfield tag="001">BV021819747</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20221129 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">061121s2004 xxud||| |||| 00||| eng d</controlfield><datafield tag="010" ind1=" " ind2=" "><subfield code="a">2003026057</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">1412909120</subfield><subfield code="c">cloth</subfield><subfield code="9">1-412-90912-0</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">141290479X</subfield><subfield code="c">pbk.</subfield><subfield code="9">1-412-90479-X</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9781412909129</subfield><subfield code="9">978-1-4129-0912-9</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9781412904797</subfield><subfield code="9">978-1-4129-0479-7</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)53814539</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV021819747</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">aacr</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="044" ind1=" " ind2=" "><subfield code="a">xxu</subfield><subfield code="c">US</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-N2</subfield><subfield code="a">DE-19</subfield><subfield code="a">DE-83</subfield><subfield code="a">DE-11</subfield><subfield code="a">DE-188</subfield><subfield code="a">DE-945</subfield><subfield code="a">DE-355</subfield><subfield code="a">DE-523</subfield><subfield code="a">DE-91</subfield></datafield><datafield tag="050" ind1=" " ind2="0"><subfield code="a">HA29</subfield></datafield><datafield tag="082" ind1="0" ind2=" "><subfield code="a">300/.72</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">CM 3000</subfield><subfield code="0">(DE-625)18945:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">CM 4000</subfield><subfield code="0">(DE-625)18951:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">PSY 470f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">SOZ 720f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">MAT 620f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">WIS 450f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Hunter, John Edward</subfield><subfield code="d">1939-2002</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)131404873</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Methods of meta-analysis</subfield><subfield code="b">correcting error and bias in research findings</subfield><subfield code="c">John E. Hunter ; Frank L. Schmidt</subfield></datafield><datafield tag="250" ind1=" " ind2=" "><subfield code="a">2. ed.</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Thousand Oaks, Calif.</subfield><subfield code="b">Sage</subfield><subfield code="c">2004</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">XXXIII, 582 S.</subfield><subfield code="b">graph. Darst.</subfield><subfield code="c">27 cm</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="500" ind1=" " ind2=" "><subfield code="a">Hier auch später erschienene, unveränderte Nachdrucke</subfield></datafield><datafield tag="520" ind1="3" ind2=" "><subfield code="a">Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic methods. Rather than taking an encyclopedic approach, the authors have focused on carefully developing those techniques that are most applicable to social science research, and have given a general conceptual description of more complex and rarely-used techniques. Fully revised and updated, the text includes new these new additions: an evaluation of fixed versus random effects models for meta-analysis; new methods for correcting for indirect range restriction in meta-analysis; new developments in corrections for measurement error; a discussion of a new Windows-based program package for applying the meta-analysis methods presented in the book; a presentation of the theories of data underlying different approaches to meta-analysis.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Ciencias sociales - Métodos estadísticos</subfield></datafield><datafield tag="650" ind1=" " ind2="7"><subfield code="a">Meta-analyse</subfield><subfield code="2">gtt</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Méta-analyse</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Sciences sociales - Méthodes statistiques</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Sozialwissenschaften</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Social sciences</subfield><subfield code="x">Statistical methods</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Meta-analysis</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Metaanalyse</subfield><subfield code="0">(DE-588)4169552-5</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Methode</subfield><subfield code="0">(DE-588)4038971-6</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Forschungsmethode</subfield><subfield code="0">(DE-588)4155046-8</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Statistik</subfield><subfield code="0">(DE-588)4056995-0</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Psychologie</subfield><subfield code="0">(DE-588)4047704-6</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Metaanalyse</subfield><subfield code="0">(DE-588)4169552-5</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="1"><subfield code="a">Forschungsmethode</subfield><subfield code="0">(DE-588)4155046-8</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="689" ind1="1" ind2="0"><subfield code="a">Metaanalyse</subfield><subfield code="0">(DE-588)4169552-5</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="1" ind2="1"><subfield code="a">Methode</subfield><subfield code="0">(DE-588)4038971-6</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="1" ind2="2"><subfield code="a">Statistik</subfield><subfield code="0">(DE-588)4056995-0</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="1" ind2="3"><subfield code="a">Psychologie</subfield><subfield code="0">(DE-588)4047704-6</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="1" ind2=" "><subfield code="8">1\p</subfield><subfield code="5">DE-604</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Schmidt, Frank L.</subfield><subfield code="d">1944-</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)131405470</subfield><subfield code="4">aut</subfield></datafield><datafield tag="785" ind1="0" ind2="0"><subfield code="i">Gefolgt von</subfield><subfield code="a">Schmidt, Frank L., 1944-</subfield><subfield code="t">Methods of meta-analysis</subfield></datafield><datafield tag="787" ind1="0" ind2="8"><subfield code="i">Überarbeitet als</subfield><subfield code="b">3. ed.</subfield><subfield code="z">978-1-4522-8689-1</subfield><subfield code="w">(DE-604)BV041790947</subfield></datafield><datafield tag="856" ind1="4" ind2=" "><subfield code="u">http://www.loc.gov/catdir/toc/ecip0411/2003026057.html</subfield><subfield code="3">Table of contents only</subfield></datafield><datafield tag="856" ind1="4" ind2=" "><subfield code="u">http://www.loc.gov/catdir/enhancements/fy0658/2003026057-d.html</subfield><subfield code="3">Publisher description</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">HBZ Datenaustausch</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=015031909&sequence=000006&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-015031909</subfield></datafield><datafield tag="883" ind1="1" ind2=" "><subfield code="8">1\p</subfield><subfield code="a">cgwrk</subfield><subfield code="d">20201028</subfield><subfield code="q">DE-101</subfield><subfield code="u">https://d-nb.info/provenance/plan#cgwrk</subfield></datafield></record></collection> |
| id | DE-604.BV021819747 |
| illustrated | Illustrated |
| index_date | 2024-07-02T15:53:47Z |
| indexdate | 2024-07-09T20:45:22Z |
| institution | BVB |
| isbn | 1412909120 141290479X 9781412909129 9781412904797 |
| language | English |
| lccn | 2003026057 |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-015031909 |
| oclc_num | 53814539 |
| open_access_boolean | |
| owner | DE-N2 DE-19 DE-BY-UBM DE-83 DE-11 DE-188 DE-945 DE-355 DE-BY-UBR DE-523 DE-91 DE-BY-TUM |
| owner_facet | DE-N2 DE-19 DE-BY-UBM DE-83 DE-11 DE-188 DE-945 DE-355 DE-BY-UBR DE-523 DE-91 DE-BY-TUM |
| physical | XXXIII, 582 S. graph. Darst. 27 cm |
| publishDate | 2004 |
| publishDateSearch | 2004 |
| publishDateSort | 2004 |
| publisher | Sage |
| record_format | marc |
| spelling | Hunter, John Edward 1939-2002 Verfasser (DE-588)131404873 aut Methods of meta-analysis correcting error and bias in research findings John E. Hunter ; Frank L. Schmidt 2. ed. Thousand Oaks, Calif. Sage 2004 XXXIII, 582 S. graph. Darst. 27 cm txt rdacontent n rdamedia nc rdacarrier Hier auch später erschienene, unveränderte Nachdrucke Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic methods. Rather than taking an encyclopedic approach, the authors have focused on carefully developing those techniques that are most applicable to social science research, and have given a general conceptual description of more complex and rarely-used techniques. Fully revised and updated, the text includes new these new additions: an evaluation of fixed versus random effects models for meta-analysis; new methods for correcting for indirect range restriction in meta-analysis; new developments in corrections for measurement error; a discussion of a new Windows-based program package for applying the meta-analysis methods presented in the book; a presentation of the theories of data underlying different approaches to meta-analysis. Ciencias sociales - Métodos estadísticos Meta-analyse gtt Méta-analyse Sciences sociales - Méthodes statistiques Sozialwissenschaften Social sciences Statistical methods Meta-analysis Metaanalyse (DE-588)4169552-5 gnd rswk-swf Methode (DE-588)4038971-6 gnd rswk-swf Forschungsmethode (DE-588)4155046-8 gnd rswk-swf Statistik (DE-588)4056995-0 gnd rswk-swf Psychologie (DE-588)4047704-6 gnd rswk-swf Metaanalyse (DE-588)4169552-5 s Forschungsmethode (DE-588)4155046-8 s DE-604 Methode (DE-588)4038971-6 s Statistik (DE-588)4056995-0 s Psychologie (DE-588)4047704-6 s 1\p DE-604 Schmidt, Frank L. 1944- Verfasser (DE-588)131405470 aut Gefolgt von Schmidt, Frank L., 1944- Methods of meta-analysis Überarbeitet als 3. ed. 978-1-4522-8689-1 (DE-604)BV041790947 http://www.loc.gov/catdir/toc/ecip0411/2003026057.html Table of contents only http://www.loc.gov/catdir/enhancements/fy0658/2003026057-d.html Publisher description HBZ Datenaustausch application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=015031909&sequence=000006&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis 1\p cgwrk 20201028 DE-101 https://d-nb.info/provenance/plan#cgwrk |
| spellingShingle | Hunter, John Edward 1939-2002 Schmidt, Frank L. 1944- Methods of meta-analysis correcting error and bias in research findings Ciencias sociales - Métodos estadísticos Meta-analyse gtt Méta-analyse Sciences sociales - Méthodes statistiques Sozialwissenschaften Social sciences Statistical methods Meta-analysis Metaanalyse (DE-588)4169552-5 gnd Methode (DE-588)4038971-6 gnd Forschungsmethode (DE-588)4155046-8 gnd Statistik (DE-588)4056995-0 gnd Psychologie (DE-588)4047704-6 gnd |
| subject_GND | (DE-588)4169552-5 (DE-588)4038971-6 (DE-588)4155046-8 (DE-588)4056995-0 (DE-588)4047704-6 |
| title | Methods of meta-analysis correcting error and bias in research findings |
| title_auth | Methods of meta-analysis correcting error and bias in research findings |
| title_exact_search | Methods of meta-analysis correcting error and bias in research findings |
| title_exact_search_txtP | Methods of meta-analysis correcting error and bias in research findings |
| title_full | Methods of meta-analysis correcting error and bias in research findings John E. Hunter ; Frank L. Schmidt |
| title_fullStr | Methods of meta-analysis correcting error and bias in research findings John E. Hunter ; Frank L. Schmidt |
| title_full_unstemmed | Methods of meta-analysis correcting error and bias in research findings John E. Hunter ; Frank L. Schmidt |
| title_new | Schmidt, Frank L., 1944- Methods of meta-analysis |
| title_short | Methods of meta-analysis |
| title_sort | methods of meta analysis correcting error and bias in research findings |
| title_sub | correcting error and bias in research findings |
| topic | Ciencias sociales - Métodos estadísticos Meta-analyse gtt Méta-analyse Sciences sociales - Méthodes statistiques Sozialwissenschaften Social sciences Statistical methods Meta-analysis Metaanalyse (DE-588)4169552-5 gnd Methode (DE-588)4038971-6 gnd Forschungsmethode (DE-588)4155046-8 gnd Statistik (DE-588)4056995-0 gnd Psychologie (DE-588)4047704-6 gnd |
| topic_facet | Ciencias sociales - Métodos estadísticos Meta-analyse Méta-analyse Sciences sociales - Méthodes statistiques Sozialwissenschaften Social sciences Statistical methods Meta-analysis Metaanalyse Methode Forschungsmethode Statistik Psychologie |
| url | http://www.loc.gov/catdir/toc/ecip0411/2003026057.html http://www.loc.gov/catdir/enhancements/fy0658/2003026057-d.html http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=015031909&sequence=000006&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT hunterjohnedward methodsofmetaanalysiscorrectingerrorandbiasinresearchfindings AT schmidtfrankl methodsofmetaanalysiscorrectingerrorandbiasinresearchfindings |