Peer Review in an Era of Evaluation : Understanding the Practice of Gatekeeping in Academia.
- 1st ed.
- 1 online resource (414 pages)
Intro -- Contents -- Notes on Contributors -- List of Figures -- List of Tables -- Part I: Peer Review: Introduction and Its Context -- 1: Peer Review in Academia -- Introduction -- The Many Face(t)s of Peer-Review Practices -- The Interplay of Primary and Secondary Peer Review -- Sites and Reviewer Tasks and Roles -- Power-making in the Selection of Expertise -- The Academic Genres of Peer Review -- Criteria and the Professional Judgement of Quality -- The Autonomy of a Profession and a Challenged Contract -- Why Study Peer Review? -- State of the Art of Research on Peer Review -- The Structure and Content of the Volume -- References -- 2: Hierarchies and Universal Inclusion in Scientific Communities -- Tensions and Contradictions in Contemporary Society -- Universal Inclusion -- Publication as the Elementary 'Unit-Act' of the System of Science -- Authorship of Publications as the Form of Inclusion in Science -- Citations as the Internal Structure of Publications -- Reading and Writing in Scientific Communities: The Hierarchy of Authorship -- The Emergence of Peer Review: The Hierarchy of Readership -- The Two Hierarchies: Authorship and Readership -- The Third and Fourth Hierarchy: Hierarchy of Publication Places and Hierarchy of Recruitment for Co-authorship -- Two Modalities of Quality Control in Science -- Anticipatory, Centralized Control by Scientific Elites -- Post-hoc, Decentralized Market Control Based on Universal Inclusion -- Concluding Remarks -- References -- 3: "'Disciplining' Educational Research in the Twentieth Century" -- Introduction -- Reviewers and Authors -- Initial Expectations -- Further Expansion -- Authors and Reviewers -- Community of Peers -- Blind Peer Review -- Papers and References -- Suggestions for Contributors -- Citation Consciousness -- Conclusion -- References. Part II: Peer Review and the Higher Education Evaluation Machinery -- 4: Gatekeepers on Campus: Peer Review in Quality Assurance of Higher Education Institutions -- Which Gates Do Peers Keep? -- Quality of Higher Education Institutions -- Quality of Performance and Performance Indicators -- Strengths and Weaknesses of Performance Indicators -- Quality of the Organisation and Peer Review -- Strengths and Weaknesses of Peer Review in Quality Assurance -- Persistent Dilemmas: Instead of a Conclusion -- References -- 5: The Many Faces of Peer Review -- Introduction -- Classical Peer Review as One of Several Evaluation Models -- Informed and Standards-Based Peer Review -- Modified Peer Review -- Extended Peer Review -- Peer Review in the Educational Field -- Conclusion and Discussion -- References -- 6: Your Brother's Gatekeeper: How Effects of Evaluation Machineries in Research Are Sometimes Enhanced -- Introduction -- Metrics and Ambiguity -- A Case Study of the Danish Bibliometric Indicator -- Vignettes -- Discussion and Conclusion -- References -- 7: Peer Review in Public Administration: The Case of the Swedish Higher Education Authority -- The National System for Quality Assurance of Higher Education in Sweden -- The History -- Critique -- 2011: A New System Is Launched -- Peer Review -- Sources -- The Role of the Peers and the Method of Peer Review -- Early Assessments 1997-1998 -- Later Assessments 2016-2017 -- Grasping the Concept of Result -- The Early Assessments 1997-1998 -- Later Assessments 2016-2017 -- Collegiality as a Form of Governance -- Peer Review in the National Quality Assurance System -- Sources -- Directions on Assessments -- Reports Describing Whole Quality Assurance Systems -- Guidelines for HEIs and Peers -- Reports on Reviews Conducted Within the Framework of National Systems for Quality Assurance of Higher Education. Accounts of Government Assignments -- Meta-analyses of the Authority's/the Authority's Own Reviews -- Other Reports -- Other -- References -- 8: Performance-Based Evaluation Metrics: Influence at the Macro, Meso, and Micro Level -- Introduction -- Algorithmic Historiography and the Birth of Scientometrics -- Bibliometrics for the Evaluation of Research -- The Citation as Mediator: The Performativity of "Being Cited" -- Consequences for Research Policy -- Co-production of Science and Society -- Performance-Based Research Allocation Models at Three Levels -- PRFS at the Macro Level: A Comparison Between the Norwegian and Swedish Systems -- At the Meso Level, Within Universities -- At the Micro Level, Individual Researchers -- Conclusions: The Performativity of Performance-Based Research Funding on Different Levels -- References -- 9: Peer Advocacy: Expressions of Loyalty in Peer Review -- Introduction -- Aims and Uses of Peer Review -- Advocacy in Academia -- Cases of Advocacy -- Case 1: National Reviews of Subjects and Programmes -- Case 2: Institutional Review of an Academic Development Unit -- Case 3: External Evaluators Appointed to Promote an Application for University Status -- Case 4: HR Competencies in Academic Appointment Processes -- Discussion -- Conclusions and Recommendations -- References -- 10: Is Peer Review Fit for Purpose? -- Introduction -- Methodology -- Refereed Journal Articles -- The Assessment of Doctoral Degrees -- The UK Research Excellence Framework (REF) -- Conclusion -- References -- Part III: Specificities of Different Peer-Review Practices -- 11: Peer Review in Academic Promotion of Excellent Teachers -- Introduction -- Career and Reward Structure in Swedish Higher Education -- Gatekeeping in the Promotion of 'Distinguished University Teachers' -- The Process of Promoting Teaching Excellence. The Mandatory Content of the Application Dossier -- Faculty Guidelines and Candidate Applications -- Judgement and Legitimation in the Making of 'Distinguished University Teachers' -- The (E)valuation of the 'Distinguished University Teacher': The Judgement -- The Scholarly Judgement of Criteria and Content -- The Scholarly Judgement of Evidence -- Justification of the (E)valuation: The Legitimation -- Justification In-between Standardization and Professional Judgement -- Justification Through Mandated Intertextuality -- Justification Through Scholarly Judgement -- The Intersection of Promotion, Peer Review, and Teaching Excellence -- The Promotion of 'Distinguished University Teachers': The Same, But Different -- A Latecomer and an Emergent Game Changer? -- References -- 12: Peers in Systematic Review: Gate Keeping Understandings of Research in the Field -- Introduction -- Peers in Academic Publishing and Systematic Review -- Analytical Framework -- Method -- Examples of Peers in three Review Processes -- Swedish Institute for Educational Research -- The Norwegian Knowledge Centre for Education -- Danish Clearinghouse for Educational Research (DCU) -- Peers in the Systematic Review Processes -- Concluding Remarks -- References -- Selected Systematic Review Technical Reports Analysed in the Study -- The Example Reports Used for the In-depth Reading Are Underlined -- 13: The Decision-Making Constraints and Processes of Grant Peer Review, and Their Effects on the Review Outcome -- Introduction -- Data Sources and Methods -- Analysis of the Grant-Review Practices of RCN -- The Various Models -- Overview of Proposals and Grants -- The Quality Criteria in the Assessments -- Considerations Other than Research Quality in the Assessments -- The Effects of the Various Ranking Processes -- Important Organizational Factors Affecting Ranking -- Conclusions. Appendix: The Review Criteria in the Different RCN Divisions -- The RCN Divisions' guidelines listed the following review criteria: -- References -- 14: Typecasting in the Recruitment of Full Professors -- Introduction -- The Field of Educational Science(s) -- Recruitment Procedures-A National Context -- Typecasting in the Recruitment of Full Professors -- The Intellectual and Social Organization of Educational Science(s) -- The Emergence of a Heterogeneous Field -- The Selection of Peers -- The Mobilization of Job Categories -- The Intersection Between Institutional Demands and Scientific Values -- Eliminatory Criteria, Signs and Retrospective Judgments -- Conclusive Notes -- Typecasting as a Device of Peer Review in the Formation of Education Science(s) -- The Professor of Education Science(s) in Late Modernity -- References -- 15: Assessing Academic Careers: The Peer Review of Professorial Candidates -- Introduction -- Picking the Best: Peer Review in Assessment Procedures -- Analysing Referee Reports -- Judgement Devices and Intellectual Structure -- Merits and Their Assessment: Five Main Themes -- Authorship, or the Reading of By-Lines -- Publication Prestige and the Importance of Articles in 'Top-Journals' -- Temporality. The Importance of a 'Positive Trajectory'. -- Racing for the Prize: Reputation Through Awards and Citations -- Boundary Keeping and the Shielding of Academic Markets -- Discussion -- References -- 16: Bureaucratic, Professional and Managerial Power in University Tenure Track Recruitment -- Introduction -- Selection and Recruitment of Candidates: A Regulative Perspective -- Recruitment and Tenure Track Systems in Finland -- Collection and Analysis of Data -- Influence and Actors in Recruitment: Bureaucratic, Managerial and Professional Powers -- The Recruitment Process in Tenure Track -- Defining the Position. Advertisement and Application.