← All judges

Andy McCallin

Trial Court · Denver · Denver

Meets Standards2 cycles on record

Evaluation history

  • 2022 retention cycle
    Trial Court · Denver · Denver
    Commission vote: 9 for · 0 against
    Meets Standards
    Retention Year: 2022 Recommendation: Meets Performance Standard Reports: 2022 Retention Survey Report 2019 Interim Survey Report Need an accessible PDF Document version? Please click on the link below and email our staff Contact Us The Second Judicial District Commission on Judicial Performance agrees by a vote of 9-0, with one Commissioner absent from voting, that Judge Andy McCallin MEETS PERFORMANCE STANDARDS. The Colorado statutory judicial performance standards are integrity, legal knowledge, communication skills, judicial temperament, administrative performance, and service to the legal profession and the public. Judge McCallin currently presides over civil cases, and previously presided over domestic relations and criminal cases. Judge McCallin meets all aspects of the performance criteria and writes in a clear, concise, and well-reasoned manner. His strengths as described in survey results include having a good judicial demeanor, ruling in a fair and impartial manner, being prepared, and being knowledgeable in the law. Based on cases open as of April 2022, Judge McCallin meets the Court’s benchmark for completing 80% of cases within 18 months. The Commission did not identify any evaluation category in which Judge McCallin displayed a notable weakness. The Commission met with the Chief Judge of Denver District Court regarding Judge McCallin’s performance, as well as with members of the District Attorney’s Office, Public Defender’s Office, and Probation Department. The Commission conducted a personal interview with Judge McCallin and observed him during court proceedings. It also reviewed opinions he authored, comments received from interested parties during the evaluation, recordings from his courtroom, case management data, and judicial performance survey responses from attorneys and non-attorneys that interacted with him. The surveys reflected personal opinions on Judge McCallin’s Case Management, Application and Knowledge of the Law, Communications, Diligence, Demeanor, and Fairness. Of the 37 attorneys and three non-attorneys who completed the survey and reported they had sufficient knowledge of Judge McCallin’s performance to evaluate him, 89% of the attorneys and 33% of the three non-attorneys believe that Judge McCallin met performance standards. Judge McCallin received overall ratings (including both attorneys and non-attorneys) at or near the statewide average among district court judges in all categories. Judge McCallin received a rating significantly above the statewide average for district court judges from the 18 appellate judges who evaluated his performance. While the Commission used these survey results in its evaluation, this evaluation tool was used in conjunction with several others to evaluate his performance. The overall data reviewed supports the Commission’s recommendation that Judge McCallin be retained.
  • 2016 retention cycle
    Trial Court · Denver · Denver
    Meets Standards(Retain)
    VotersRetained· 76.5% yes
    168,278 yes · 51,669 no50% needed to retain
    Retention Year: 2016 Recommendation: Meets Performance Standard Reports: 2016 Retention Survey Report (PDF) 2015 Interim Survey Report (PDF) Need an accessible PDF Document Version? Please click on the link below and email our staff Contact Us The Second Judicial District Commission on Judicial Performance, on a vote of 10-0, unanimously recommends that Judge Andy McCallin BE RETAINED. Governor John Hickenlooper appointed Judge McCallin to the Denver District Court in January 2014. Before his appointment, Judge McCallin served as an assistant attorney general in the consumer protection division and as an adjunct professor at the University of Denver Sturm College of Law. Previously, while in private practice, he specialized in complex commercial litigation. Judge McCallin received his bachelor's degree from Claremont McKenna College and his law degree from the University of Denver Sturm College of Law. Judge McCallin continues to lecture at the University of Denver Sturm College of Law. The Commission conducted a personal interview with Judge McCallin, reviewed opinions he authored, his self-evaluation, observed him in court, and reviewed survey responses from attorneys and non-attorneys who had experience with Judge McCallin. Of attorneys completing the survey, 96% recommended retaining and 4% not to retain. Of non-attorneys completing the survey, 97% recommended to retain and 3% not to retain. 23 attorneys and 32 non-attorneys completed  the survey. In his limited time on the bench, Judge McCallin has excelled. Survey results reflect overall confidence from appellate judges and attorneys. Non-attorneys scored Judge McCallin above the average district court judge in all areas. Of note, Judge McCallin scored high marks in the areas of demeanor, application of the law, and fairness from both attorneys and non-attorneys. Commissioners noted that during courtroom observations that Judge McCallin appears to have an even demeanor and oversees cases with a high degree of impartiality. Given his strengths, the Commission is confident in recommending the voters retain Judge McCallin.

Surveys & reports

  • 2022 cycle
    • 2021 Interim Survey Report(PDF on OJPE)
      Read full interim survey textexpand
      ## Page 1 The Honorable Judge Andrew P. McCallin 2019 Judicial Performance Survey Report District Court Conducted by: 75 Washington Ave. Ste. 2C Portland, ME 04101 www.marketdecisions.com ## Page 2 Contents Summary of Results . 3 Overall Score . 3 Performance Scores . 4 Individual Category Scores . 5 Summary of Responses . 6 Detailed Report . 7 Case Management . 7 Application and Knowledge of Law . 8 Communications . 9 Demeanor .10 Diligence .11 Fairness .13 Survey of Appellate Judges .14 Appendix 1. Survey Methods – Attorney and Non-attorney .15 Methodology and How to Read Results .15 Appendix 2. Survey Methodology - Appellate Judges.18 Methodology and How to Read Results .18 Appendix 3: Judge Response Counts by Type of Respondent .19 i ## Page 3 Summary of Results For Judge McCallin, 79 individuals completed surveys with at least a single rating question answered. This report reflects these 79 responses. Respondents rated judges on various questions using an A to F scale, in which the grades were then converted to the following numerical scores: A= 4, B=3, C=2, D=1 and Fail=0. An average score of 4.0 is the highest possible score and a 0.0 is the lowest possible score. Overall Score 4 3.4 2.9 3 2 1 0 Judge McCallin All District Judges Judge McCallin All District Judges Overall Grade 2.9 3.4 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 3 ## Page 4 Performance Scores 100% 85% 85% 82% 80% 62% 60% 40% 29% 20% 15% 12% 11% 9% 6% 3% 0% Judge McCallin All District Judges Judge McCallin All District Judges Attorneys Non-Attorneys Yes, meets performance standards No, does not meet performance standards No opinion Attorneys Non-Attorneys All All Judge Judge District District McCallin McCallin Judges Judges Yes, meets performance 62% 82% 85% 85% standards No, does not meet 29% 12% 15% 11% performance standards No opinion 9% 6% 0% 3% Note: All percentages in this report are rounded to the nearest percentage point. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 4 ## Page 5 Individual Category Scores 0.0 1.0 2.0 3.0 4.0 2.8 Case Management 3.4 2.7 Application and Knowledge of Law 3.4 3.0 Communications 3.6 2.7 Diligence 3.4 2.9 Demeanor 3.5 3.3 Fairness 3.5 Judge McCallin All District Judges Judge All District McCallin Judges Case 2.8 3.4 Management Application and Knowledge of 2.7 3.4 Law Communications 3.0 3.6 Diligence 2.7 3.4 Demeanor 2.9 3.5 Fairness 3.3 3.5 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 5 ## Page 6 Summary of Responses Percent Number Response with with Group Responses Rate Sufficient Sufficient Knowledge Knowledge Attorneys 119 49% 55% 66 Non- 17 3% 76% 13 Attorneys In addition to the responses above, Judge McCallin received 0 response(s) via the open Citizen Feedback survey. Those responses are included with non-attorney results wherever applicable. However, due to the nature of data collection, they are not included in response rates. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 6 ## Page 7 Detailed Report Case Management 0 1 2 3 4 Judge McCallin Overall 2.8 District Judges Overall 3.4 Judge McCallin Attorneys 2.8 Judge McCallin Non-Attorneys Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Overall Overall Attorneys Attorneys Promptly issuing a decision on 3 3.4 3 N/A 61 the case after trial Maintaining appropriate control 2.8 3.4 2.8 N/A 68 over proceedings Promptly ruling on pre-trial 2.9 3.3 2.9 N/A 59 motions Setting reasonable schedules for 2.8 3.3 2.8 N/A 66 cases 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 7 ## Page 8 Application and Knowledge of Law 0 1 2 3 4 Judge McCallin Overall 2.7 District Judges Overall 3.4 Judge McCallin Attorneys 2.6 Judge McCallin Non-Attorneys 3.4 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Overall Overall Attorneys Attorneys Being able to identify and 2.8 3.4 2.7 3.3 78 analyze relevant facts Basing decisions on evidence 2.5 3.2 2.5 N/A 65 and arguments Issuing consistent sentences when the circumstances are 1.9 3.1 1.9 N/A 31 similar Being fair and impartial to both 2.7 3.3 2.7 N/A 66 sides of the case Consistently applying laws and 2.5 3.2 2.5 N/A 65 rules Giving reasons for rulings 3.4 3.5 N/A 3.4 12 Willing to make decision without regard to possible outside 3.5 3.5 N/A 3.5 11 pressure 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 8 ## Page 9 Communications 0 1 2 3 4 Judge McCallin Overall 3 District Judges Overall 3.6 Judge McCallin Attorneys 2.9 Judge McCallin Non-Attorneys 3.8 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Overall Overall Attorneys Attorneys Making sure all participants 3.1 3.6 3 3.7 78 understand the proceedings Providing written communications that are clear, 2.7 3.3 2.7 N/A 62 thorough and well reasoned Using language that everyone 3.7 3.7 N/A 3.7 11 can understand Speaking clearly so everyone in the courtroom can hear what’s 3.8 3.7 N/A 3.8 11 being said 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 9 ## Page 10 Demeanor 0 1 2 3 4 Judge McCallin Score 2.9 District Judges Overall 3.5 Judge McCallin Attorneys 2.8 Judge McCallin Non-Attorneys 3.4 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Score Overall Attorneys Attorneys Giving proceedings a sense of dignity 3 3.5 2.9 3.6 80 Treating participants with respect 3 3.5 2.9 3.4 79 Conducting his/her courtroom in a 2.8 3.4 2.7 3.3 77 neutral manner Having a sense of compassion and human understanding for those who 3.3 3.5 N/A 3.3 11 appear before him/her 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 10 ## Page 11 Diligence 0 1 2 3 4 Judge McCallin Score 2.7 District Judges Overall 3.4 Judge McCallin Attorneys 2.6 Judge McCallin Non-Attorneys 3.5 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Score Overall Attorneys Attorneys Using good judgment in application of relevant law and 2.6 3.2 2.6 N/A 66 rules Doing the necessary “homework” and being prepared 2.8 3.5 2.6 3.5 77 for cases Being willing to handle cases on the docket even when they are 2.6 3.4 2.6 N/A 62 complicated and time consuming Beginning court on time 3.5 3.6 N/A 3.5 11 Maintaining appropriate control 3.5 3.7 N/A 3.5 10 over proceedings Setting reasonable schedules for 3.3 3.6 N/A 3.3 10 cases Managing court proceedings so 3.3 3.5 N/A 3.3 10 that there is little wasted time 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 11 ## Page 12 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 12 ## Page 13 Fairness 0.0 1.0 2.0 3.0 4.0 Judge McCallin Score 3.3 District Judges Overall 3.5 Judge McCallin Attorneys Judge McCallin Non-Attorneys 3.3 Judge Judge Judge District Number of McCallin McCallin Non- McCallin Score Judges Overall Responses Attorneys Attorneys Giving participants an 3.6 3.6 N/A 3.6 12 opportunity to be heard Treating those involved 3.3 3.5 N/A 3.3 12 in the case without bias Treating fairly people who represent 3.4 3.6 N/A 3.4 7 themselves Giving each side enough time to present his or her 3.2 3.5 N/A 3.2 11 case 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 13 ## Page 14 Survey of Appellate Judges For Judge McCallin 12 appellate judges agreed they had worked with Judge McCallin enough to evaluate their performance. 0.0 1.0 2.0 3.0 4.0 Judge McCallin Score 3.7 Judge McCallin District Judges Number of Score Overall Responses Overall performance as a judge 3.7 3.6 12 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 14 ## Page 15 Appendix 1. Survey Methods – Attorney and Non- attorney Methodology and How to Read Results For Judge McCallin, 79 individuals completed surveys with at least a single rating question answered. This report reflects these 79 responses. The survey results are divided into eight sections: Retention, Case Management, Application and Knowledge of Law, Communications, Demeanor, Diligence, Fairness, Strengths, and Weaknesses. a. Response rates Attorneys During the 2018 administration, a total of 16,364 survey invitations were sent to 5,769 attorneys inviting them to evaluate judges and justices receiving interim reports in 2019. On average, each attorney was asked to evaluate 2.8 judges. In total 3,837 surveys were completed with an additional 2,309 responses where the attorney indicated that they did not have enough experience with the judge to be comfortable evaluating him or her. The response rate for the survey was 38% and the survey completion rate (the number of those familiar enough to evaluate the judge divided by the total number of attorney responses including those indicating they did not have sufficient familiarity to evaluate the judge) was 62%. Non-attorneys During the 2018 administration, a total of 143,430 survey invitations were sent to non-attorneys inviting them to evaluate judges and justices receiving interim reports in 2019. In total 4,862 surveys were completed with an additional 1,925 responses where the individual indicated that they did not have enough experience with the judge to be comfortable evaluating him or her. The response rate for the survey was 6% and the survey completion rate (the number of those familiar enough to evaluate the judge divided by the total number of responses including those indicating they did not have sufficient familiarity to evaluate the judge) was 72%. b. Methodology The 2018 attorney survey was conducted in 4 quarterly cycles online beginning on June 4th, 2018. Attorneys with appearances in front of judges during the first quarter of 2018 were first mailed a pre-notification letter on June 4th, 2018 informing them about the survey and providing a link and login information to access the survey online. Next, a series of three email invitations were sent on June 7th, June 18th, and July 2nd, 2018. This process was repeated among attorneys with appearances in the second quarter of 2018 beginning with a pre- notification letter sent on August 10th, 2018. The letter was followed up by email invitations sent on August 13th, August 27th, and September 11th, 2018. For those attorneys with appearances during the 3rd quarter of 2018, the process was repeated again with a letter sent on November 26th and a series of emails sent on November 29th, December 11th, and December 29th, 2018. To further increase the amount of data collected, an additional cycle of data collection took place in January and February 2019. Invitations were emailed to attorneys with appearances during the 4th quarter of 2018. This cycle included a pre-notification letter sent on January 25th followed up by email invitations sent on February 5th, February 11th, and February 19th, 2019. Additional invitations and reminders were sent out on request throughout the data collection process. Data collection among non-attorneys also featured four quarterly cycles with each cycle consisting of a mailed invitation letter with instructions to access the survey online followed by a mailed survey booklet with a prepaid 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 15 ## Page 16 return envelope. To meet tight reporting deadlines, the order was switched for the 4th quarter administration and the survey booklet was sent first followed by the letter. Due to the ability to contact court staff via email, respondents were split into two groups for data collection: court staff and other non-attorneys. The court staff group includes staff members, interpreters and probation officers. The other non-attorney group includes jurors, defendants, witnesses, litigants, and law enforcement personnel. Court staff members were invited via emailed invitations sent on November 5th, 2018 and reminders sent on November 13th and 26th. In addition to the main non-attorney survey where respondents were invited to participate, there was also a citizen feedback survey available to all citizens on the OJPE website. This survey was open for the entire data collection period and data was downloaded for analysis on February 21st, 2019. During this period 87 valid responses were received. The survey remained open and any responses received after February 21stor for judges not receiving an interim evaluation in 2019 were held over for the 2020 evaluation cycle. Table 1: Non-Attorney Data Collection Dates Cycle Invitation Letter Survey Booklet Q1 5/18/18 6/20/18 Q2 8/2/18 9/7/18 Q3 11/9/18 12/19/18 Q4 1/31/19 1/23/19 c. Questions In the core of the survey, attorneys evaluated district and county judges on 17 aspects of judicial performance and appellate judges on 12 aspects of judicial performance using a grade scale of A, B, C, D, or F. These aspects were grouped by topic into different categories, five for district and county judges and two for appellate judges. The district and county categories were: Case Management, Application and Knowledge of Law, Communications, Demeanor, and Diligence. Questions regarding appellate judges were divided into two categories, one for general questions and one specific to their writing (only asked of those who indicated they had experience with the judge or justice’s written opinions). In a final question, respondents were asked if they thought whether the judge met judicial performance standards The question wording for the core of the survey was carried over from the 2017 administration with only minor changes to make the survey gender neutral. The questions were originally developed in 1998 to meet the criteria outlined in statute 13-5.5-101 et seq. Non-attorney respondents evaluated judges on 19 aspects of judicial performance using the same grade scale of A, B, C, D, or Fail. In a final question, respondents were asked if they thought whether the judge met judicial performance standards. The overall structure of the survey was similar to the attorney survey, but the individual rating questions were tailored to aspects that could be rated by those without specific legal experience. d. Analysis and Reporting Letter grades were converted to a numerical score where A = 4, B = 3, C = 2, D = 1 and Fail = 0 for analysis. The results include an overall grade, a grade for each category, as well as a grade for each question. The overall score is calculated by averaging the responses to all questions answered by the attorneys. This score will have the same numerical range as the individual questions from zero to four. Each category score is calculated by averaging the responses to all questions answered by the attorney within each category. This score will have the same zero to four numerical range as the individual questions. Similarly, 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 16 ## Page 17 an average score is calculated for each individual question with the exception of the final question on meeting performance standards. The overall average and category scores will be reported for each judge along with the average scores for the judge’s peers. The average score (with the exception noted above) will also be reported for each question along with the peer group score. In addition, the report will include the distribution of responses for each question, i.e. the percentage of attorneys that assigned a rating of A, B, C, D, and F. The distribution of responses is also reported for the question on retention. e. Comments At the end of each group of questions respondents had the option of leaving comments about the judge’s performance in that area. By statute, these comments are confidential and only provided to the judge and the District Commission on Judicial Performance. They are not released to the public when the rest of the report is released. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 17 ## Page 18 Appendix 2. Survey Methodology - Appellate Judges Methodology and How to Read Results a. Response rates Invitations were sent via email to all 29 Supreme Court justices and Court of Appeals judges. Of these, 12 completed the survey and felt they had sufficient knowledge of Judge McCallin to evaluate their performance. b. Methodology Appellate judges were surveyed to evaluate the performance of district judges. This evaluation of district judges was conducted via an online survey hosted in the Voxco survey software. An email invitation was sent on January 29th, 2019. c. Questions Due to the large number of judges being evaluated, the district judge evaluation survey consisted of a single question pertaining to each judge. Appellate judges and justices were asked to evaluate the district judge’s overall performance as a judge on a grade scale of A-F with A being “Excellent” and F being “Fail”. In the survey, the district judges being evaluated were grouped by district with the districts presented in random order to reduce bias. d. Analysis and Reporting Letter grades were then converted to a numerical score where A = 4, B = 3, C = 2, D = 1 and Fail = 0 for analysis. The overall score is calculated by averaging the responses to all questions answered. This score will have the same numerical range as the individual questions from zero to four. The overall average will be reported for each judge along with the average scores for the judge’s peers. In addition, the report will include the distribution of responses for each question. That is, the percentage of respondents that assigned a rating of A, B, C, D, and F. e. Comments Respondents were given the option to leave supporting comments in a box next to where they graded each judge. By statute, these comments are confidential and only provided to the judge and the District Commission on Judicial Performance. They are not released to the public when the rest of the report is released. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 18 ## Page 19 Appendix 3: Judge Response Counts by Type of Respondent Number of Undeliverable/ Cooperation Respondent Type Total Sent Completes* Responses Not Applicable Rate Attorney 245 119 0 66 55% Law Enforcement 19 0 1 0 - Litigant 557 9 135 6 67% Juror 0 0 0 0 - Others 164 8 17 7 88% Total Non-Attorneys 740 17 153 13 76% Total Respondents 985 136 153 79 58% *Completed surveys include respondents who said that they have sufficient experience to evaluate the judge. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 19
    • 2022 Retention Survey Report(PDF on OJPE)
      Overall score
      3.3
      judge rating, 0–4 scale
      Attorneys recommend
      89%
      110 respondents
      Non-attorneys recommend
      33%
      respondent count not on record
      Read full retention survey textexpand
      ## Page 1 The Honorable Judge Andrew P. McCallin 2022 Judicial Performance Survey Report District Court Conducted by: 511 Congress Street, Suite 801 Portland, ME 04101 www.marketdecisions.com ## Page 2 Contents Summary of Results . 3 Overall Score . 3 Performance Scores . 4 Individual Category Scores . 5 Summary of Responses . 6 Detailed Report . 8 Case Management . 8 Application and Knowledge of Law . 9 Communications . 10 Demeanor . 11 Diligence . 12 Fairness . 13 Survey of Appellate Judges . 14 Appendix 1. Survey Methods – Attorney and Non-attorney . 15 Methodology and How to Read Results . 15 Appendix 2. Survey Methodology - Appellate Judges . 18 Methodology and How to Read Results . 18 Appendix 3: Judge Response Counts by Type of Respondent . 19 i ## Page 3 Summary of Results For Judge McCallin, 40 individuals completed surveys with at least a single rating question answered. This report reflects these 40 responses. Respondents rated judges on various questions using an A to F scale, in which the grades were then converted to the following numerical scores: A= 4, B=3, C=2, D=1 and Fail=0. An average score of 4.0 is the highest possible score and a 0.0 is the lowest possible score. Overall Score 4 3.4 3.3 3 2 1 0 Judge McCallin All District Judges Judge McCallin All District Judges Overall Grade 3.3 3.4 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 3 ## Page 4 Performance Scores 100% 89% 83% 79% 80% 67% 60% 40% 33% 20% 16% 12% 6% 6% 4% 5% 0% Judge McCallin All District Judges Judge McCallin All District Judges Attorneys Non-Attorneys Yes, meets performance standards No, does not meet performance standards No opinion Attorneys Non-Attorneys All All Judge Judge District District McCallin McCallin Judges Judges Yes, meets performance 89% 83% 33% 79% standards No, does not meet 6% 12% 67% 16% performance standards No opinion 6% 4% 0% 5% Note: All percentages in this report are rounded to the nearest percentage point. 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 4 ## Page 5 Individual Category Scores 0.0 1.0 2.0 3.0 4.0 3.3 Case Management 3.4 3.1 Application and Knowledge of Law 3.3 3.5 Communications 3.5 3.2 Diligence 3.4 3.5 Demeanor 3.4 3.4 Fairness 3.3 Judge McCallin All District Judges Judge All District McCallin Judges Case Management 3.3 3.4 Application and Knowledge 3.1 3.3 of Law Communications 3.5 3.5 Diligence 3.2 3.4 Demeanor 3.5 3.4 Fairness 3.4 3.3 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 5 ## Page 6 Summary of Responses Percent Number Response with with Group Responses Rate Sufficient Sufficient Knowledge Knowledge Attorneys 110 23% 34% 37 Non- 5 - - 3 Attorneys *Some non-attorney evaluations are self-selected, making calculating a response rate impossible. The non-attorney group includes staff, jurors, litigants, and witnesses including law enforcement officers. It also includes responses provided through the open Citizen feedback survey. Judge McCallin received 0 response(s) via the Citizen Feedback survey. Those responses are included with non- attorney results wherever applicable. However, as self-selected evaluations they cannot be included in response rates. 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 6 ## Page 7 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 7 ## Page 8 Detailed Report Case Management 0 1 2 3 4 Judge McCallin Overall 3.3 District Judges Overall 3.4 Judge McCallin Attorneys 3.4 Judge McCallin Non-Attorneys 1.3 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Overall Overall Attorneys Attorneys Promptly issuing a decision on 3.1 3.4 3.3 1.7 27 the case after trial Maintaining appropriate control 3.4 3.4 3.5 1.5 38 over proceedings Promptly ruling on pre-trial 3.2 3.3 3.3 2.0 36 motions Setting reasonable schedules for 3.3 3.4 3.6 0.3 39 cases 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 8 ## Page 9 Application and Knowledge of Law 0 1 2 3 4 Judge McCallin Overall 3.1 District Judges Overall 3.3 Judge McCallin Attorneys 3.3 Judge McCallin Non-Attorneys 1.2 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Overall Overall Attorneys Attorneys Being able to identify and 3.2 3.3 3.4 1.3 37 analyze relevant facts Basing decisions on evidence 3.3 3.2 3.3 N/A 34 and arguments Issuing consistent sentences when the circumstances are 3.4 3.3 3.4 N/A 27 similar Consistently applying laws and 3.4 3.3 3.4 N/A 32 rules Giving reasons for rulings 1.3 3.2 N/A 1.3 3 Willing to make decisions without regard to possible 1 3.3 N/A 1.0 3 outside pressure 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 9 ## Page 10 Communications 0 1 2 3 4 Judge McCallin Overall 3.5 District Judges Overall 3.5 Judge McCallin Attorneys 3.6 Judge McCallin Non-Attorneys 2.6 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Overall Overall Attorneys Attorneys Making sure all participants 3.6 3.5 3.7 2.0 35 understand the proceedings Providing written communications that are clear, 3.5 3.3 3.5 N/A 33 thorough and well-reasoned Using language that everyone 3 3.6 N/A 3.0 2 can understand Speaking clearly so everyone in the courtroom can hear what’s 3 3.6 N/A 3.0 3 being said 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 10 ## Page 11 Demeanor 0 1 2 3 4 Judge McCallin Score 3.5 District Judges Overall 3.4 Judge McCallin Attorneys 3.6 Judge McCallin Non-Attorneys 2.1 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Score Overall Attorneys Attorneys Giving proceedings a sense of dignity 3.6 3.5 3.7 2.3 38 Treating participants with respect 3.4 3.4 3.5 2.3 38 Having a sense of compassion and human understanding for those who 1.7 3.2 N/A 1.7 3 appear before him/her 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 11 ## Page 12 Diligence 0 1 2 3 4 Judge McCallin Score 3.2 District Judges Overall 3.4 Judge McCallin Attorneys 3.4 Judge McCallin Non-Attorneys 1.6 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Score Overall Attorneys Attorneys Using good judgment in application of relevant law and 3.3 3.2 3.3 N/A 33 rules Being willing to handle cases on the docket even when they are 3.4 3.4 3.4 N/A 29 complicated and time consuming Doing the necessary “homework” and being prepared 3.4 3.4 3.4 N/A 33 for cases Beginning court on time. 4 3.5 N/A 4.0 2 Maintaining appropriate control 2 3.5 N/A 2.0 2 over proceedings Setting reasonable schedules for 0.3 3.4 N/A 0.3 3 cases Being prepared for cases. 1.7 3.4 N/A 1.7 3 Managing court proceedings so 2 3.4 N/A 2.0 2 that there is little wasted time 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 12 ## Page 13 Fairness 0.0 1.0 2.0 3.0 4.0 Judge McCallin Score 3.4 District Judges Overall 3.3 Judge McCallin Attorneys 3.5 Judge McCallin Non-Attorneys 1.5 Judge Judge Judge District Number of McCallin McCallin Non- McCallin Score Judges Overall Responses Attorneys Attorneys Giving participants an 1.0 3.3 N/A 1.0 3 opportunity to be heard Treating those involved 2.3 3.2 N/A 2.3 3 in the case without bias Treating fairly people who represent 1.0 3.3 N/A 1.0 1 themselves Giving each side enough time to present his or her 1.0 3.3 N/A 1.0 3 case Conducting their courtroom in a neutral 3.4 3.3 3.6 2.0 38 manner. Being fair and impartial 3.5 3.3 3.5 N/A 35 to both sides. 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 13 ## Page 14 Survey of Appellate Judges For Judge McCallin 18 appellate judges agreed they had worked with Judge McCallin enough to evaluate their performance. 0.0 1.0 2.0 3.0 4.0 Judge McCallin Score 3.9 District Judges Overall 3.7 Judge McCallin District Judges Number of Score Overall Responses Overall performance as a judge 3.9 3.7 18 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 14 ## Page 15 Appendix 1. Survey Methods – Attorney and Non- attorney Methodology and How to Read Results For Judge McCallin, 40 individuals completed surveys with at least a single rating question answered. This report reflects these 40 responses. The survey results are divided into eight sections: Case Management, Application and Knowledge of Law, Communications, Demeanor, Diligence, Fairness, Strengths, and Weaknesses. a. Response rates Attorneys During the administration, a total of 89,078 survey invitations were sent to 10,845 attorneys inviting them to evaluate judges and justices receiving reports in 2022. On average, each attorney was asked to evaluate 8.2 judges. In total 9,005 surveys were completed with an additional 9,895 responses where the attorney indicated that they did not have enough experience with the judge to be comfortable evaluating him or her. The response rate for the survey was 10% and the survey completion rate (the number of those familiar indicating they did not have sufficient familiarity to evaluate the judge) was 21%. Non-attorneys The 2022 administration expanded on contacting non-attorneys electronically. More non-attorney email addresses became available due to administrative changes in the courts. In total 86,758 non-attorneys were invited via email. Some non-attorney groups could self-select which judges they would like to evaluate. Court staff members were emailed a link that allowed them to self-select the judges wished to evaluate in their county or district. Jurors, in addition to email, could similarly self-select via publicly posted links. The public was also allowed to self-select via a separate public link. These methods allowed us to gather more data than previous cycles, however the inclusion of self-selected evaluations makes it impossible to calculate a response rate. In total 5,463 surveys were completed with an additional 2,466 responses where the respondent indicated that they did not have enough experience with the judge to be comfortable providing an evaluation. b. Methodology The 2022 attorney survey was conducted in 4 cycles online beginning on June 15th, 2021. Attorneys with appearances in front of judges in each quarter were sent a sent a series of email invitations. Invitations were emailed to attorneys with appearances during the first quarter of 2021 on June 15th, 2021. Reminders were sent on June 30th and July 9th, 2021. This process was repeated among attorneys with appearances in the second quarter of 2021 with email invitations sent on August 23rd, 2021 and reminders was sent on September 2nd, 2021 and September 14th, 2021. Invitations were emailed to attorneys with appearances during the third quarter of 2021 on November 9th, 2021. Reminders sent on November 18th and December 2nd, 2021. The final data collection took place in January 2022. Invitations were emailed to attorneys with appearances during the 4th quarter of 2021 on January 19th, 2022. Reminders were sent on January 28th, February 2nd and 12th, 2022. Invitations were sent out on request throughout the data collection process. 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 15 ## Page 16 Data collection for non-attorneys began on January 1st, 2021 and ran through the response deadline of February 28th, 2022. Survey invitations were sent via email to most non-attorneys in quarterly batches mirroring the process used for attorneys. Court staff members were also invited via email but using a different process. To help reduce administrative burden, the way that court staff were invited was changed for this cycle. Rather than be invited to evaluate specific judges, staff are now brought to a screen showing list of all judges in their district our county and allowed to choose which to evaluate. Jurors, in addition to email, were allowed to similarly self-select via publicly posted links. The general public was also allowed to self-select via a separate public link. This survey was open for the entire data collection period and data was downloaded for analysis on February 28th, 2022. During this period 938 valid responses were received. The survey remained open and any responses received after February 28th or for judges not receiving an evaluation in 2022 were held over for the 2023 evaluation cycle. Details on the responses from each group are detailed in the table below. Table 1: Non-Attorney completes by invitation method Invitation Method Invites Sent Completes Response Rate Non-attorney email 90,845 6,068 7% Court Staff Email Invite 3,471 1,861 N/A (Self-select) Citizen Feedback Unknown 220 N/A Juror Survey Unknown 938 N/A c. Questions In the core of the survey, attorneys evaluated district and county judges on 17 aspects of judicial performance and appellate judges on 12 aspects of judicial performance using a grade scale of A, B, C, D, or F. These aspects were grouped by topic into different categories, five for district and county judges and two for appellate judges. The district and county categories were: Case Management, Application and Knowledge of Law, Communications, Demeanor, Diligence and fairness. Questions regarding appellate judges were divided into two categories, one for general questions and one specific to their writing (only asked of those who indicated they had experience with the judge or justice’s written opinions). In a final question, respondents were asked if they thought whether the judge met judicial performance standards The question wording for the core of the survey was carried over from the 2021 administration. The questions were originally developed in 1998 to meet the criteria outlined in statute 13-5.5-101 et seq. Non-attorney respondents evaluated judges on 23 aspects of judicial performance using the same grade scale of A, B, C, D, or Fail. In a final question, respondents were asked if they thought whether the judge met judicial performance standards. The overall structure of the survey was similar to the attorney survey, but the individual rating questions were tailored to aspects that could be rated by those without specific legal experience. d. Analysis and Reporting Letter grades were converted to a numerical score where A = 4, B = 3, C = 2, D = 1 and Fail = 0 for analysis. The results include an overall grade, a grade for each category, as well as a grade for each question. The overall score is calculated by averaging the responses to all questions answered by the attorneys. This score will have the same numerical range as the individual questions from zero to four. Each category score is calculated by averaging the responses to all questions answered by the attorney within each category. This score will have the same zero to four numerical range as the individual questions. Similarly, 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 16 ## Page 17 an average score is calculated for each individual question with the exception of the final question on meeting performance standards. The overall average and category scores will be reported for each judge along with the average scores for the judge’s peers. The average score (with the exception noted above) will also be reported for each question along with the peer group score. In addition, the report will include the distribution of responses for each question, i.e. the percentage of attorneys that assigned a rating of A, B, C, D, and F. The distribution of responses is also reported for the question on retention. e. Comments At the end of each group of questions respondents had the option of leaving comments about the judge’s performance in that area. By statute, these comments are confidential and only provided to the judge and the District Commission on Judicial Performance. They are not released to the public when the rest of the report is released. 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 17 ## Page 18 Appendix 2. Survey Methodology - Appellate Judges Methodology and How to Read Results a. Response rates Invitations were sent via email to all 39 Supreme Court justices and Court of Appeals judges. Of these, 18 completed the survey and felt they had sufficient knowledge of Judge McCallin to evaluate their performance. b. Methodology Appellate judges were surveyed to evaluate the performance of district judges standing for retention. This evaluation of district judges was conducted via an online survey hosted in the Voxco survey software. An email invitation was sent on January 21st, 2022. c. Questions Due to the large number of judges being evaluated, the district judge evaluation survey consisted of a single question pertaining to each judge. Appellate judges and justices were asked to evaluate the district judge’s overall performance as a judge on a grade scale of A-F with A being “Excellent” and F being “Fail”. In the survey, the district judges being evaluated were grouped by district with the districts presented in random order to reduce bias. d. Analysis and Reporting Letter grades were then converted to a numerical score where A = 4, B = 3, C = 2, D = 1 and Fail = 0 for analysis. The overall score is calculated by averaging the responses to all questions answered. This score will have the same numerical range as the individual questions from zero to four. The overall average will be reported for each judge along with the average scores for the judge’s peers. In addition, the report will include the distribution of responses for each question. That is, the percentage of respondents that assigned a rating of A, B, C, D, and F. e. Comments Respondents were given the option to leave supporting comments in a box next to where they graded each judge. By statute, these comments are confidential and only provided to the judge and the District Commission on Judicial Performance. They are not released to the public when the rest of the report is released. 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 18 ## Page 19 Appendix 3: Judge Response Counts by Type of Respondent Number of Cooperation Respondent Type Total Sent Completes* Responses Rate Attorney 486 110 37 34% Staff - 0 0 - Others 13 5 3 60% Juror - 0 0 - Citizen Feedback - 0 0 Total Non-Attorneys - 5 3 - Total Respondents - 115 40 - *Completed surveys include respondents who said that they have sufficient experience to evaluate the judge. 2022 Judicial Performance Survey Report for Judge Andrew P. McCallin 19
  • 2016 cycle
    • 2015 Interim Survey Report(PDF on OJPE)
      Read full interim survey textexpand
      ## Page 1 The Honorable Judge Andrew P. McCallin 2019 Judicial Performance Survey Report District Court Conducted by: 75 Washington Ave. Ste. 2C Portland, ME 04101 www.marketdecisions.com ## Page 2 Contents Summary of Results . 3 Overall Score . 3 Performance Scores . 4 Individual Category Scores . 5 Summary of Responses . 6 Detailed Report . 7 Case Management . 7 Application and Knowledge of Law . 8 Communications . 9 Demeanor .10 Diligence .11 Fairness .13 Survey of Appellate Judges .14 Appendix 1. Survey Methods – Attorney and Non-attorney .15 Methodology and How to Read Results .15 Appendix 2. Survey Methodology - Appellate Judges.18 Methodology and How to Read Results .18 Appendix 3: Judge Response Counts by Type of Respondent .19 i ## Page 3 Summary of Results For Judge McCallin, 79 individuals completed surveys with at least a single rating question answered. This report reflects these 79 responses. Respondents rated judges on various questions using an A to F scale, in which the grades were then converted to the following numerical scores: A= 4, B=3, C=2, D=1 and Fail=0. An average score of 4.0 is the highest possible score and a 0.0 is the lowest possible score. Overall Score 4 3.4 2.9 3 2 1 0 Judge McCallin All District Judges Judge McCallin All District Judges Overall Grade 2.9 3.4 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 3 ## Page 4 Performance Scores 100% 85% 85% 82% 80% 62% 60% 40% 29% 20% 15% 12% 11% 9% 6% 3% 0% Judge McCallin All District Judges Judge McCallin All District Judges Attorneys Non-Attorneys Yes, meets performance standards No, does not meet performance standards No opinion Attorneys Non-Attorneys All All Judge Judge District District McCallin McCallin Judges Judges Yes, meets performance 62% 82% 85% 85% standards No, does not meet 29% 12% 15% 11% performance standards No opinion 9% 6% 0% 3% Note: All percentages in this report are rounded to the nearest percentage point. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 4 ## Page 5 Individual Category Scores 0.0 1.0 2.0 3.0 4.0 2.8 Case Management 3.4 2.7 Application and Knowledge of Law 3.4 3.0 Communications 3.6 2.7 Diligence 3.4 2.9 Demeanor 3.5 3.3 Fairness 3.5 Judge McCallin All District Judges Judge All District McCallin Judges Case 2.8 3.4 Management Application and Knowledge of 2.7 3.4 Law Communications 3.0 3.6 Diligence 2.7 3.4 Demeanor 2.9 3.5 Fairness 3.3 3.5 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 5 ## Page 6 Summary of Responses Percent Number Response with with Group Responses Rate Sufficient Sufficient Knowledge Knowledge Attorneys 119 49% 55% 66 Non- 17 3% 76% 13 Attorneys In addition to the responses above, Judge McCallin received 0 response(s) via the open Citizen Feedback survey. Those responses are included with non-attorney results wherever applicable. However, due to the nature of data collection, they are not included in response rates. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 6 ## Page 7 Detailed Report Case Management 0 1 2 3 4 Judge McCallin Overall 2.8 District Judges Overall 3.4 Judge McCallin Attorneys 2.8 Judge McCallin Non-Attorneys Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Overall Overall Attorneys Attorneys Promptly issuing a decision on 3 3.4 3 N/A 61 the case after trial Maintaining appropriate control 2.8 3.4 2.8 N/A 68 over proceedings Promptly ruling on pre-trial 2.9 3.3 2.9 N/A 59 motions Setting reasonable schedules for 2.8 3.3 2.8 N/A 66 cases 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 7 ## Page 8 Application and Knowledge of Law 0 1 2 3 4 Judge McCallin Overall 2.7 District Judges Overall 3.4 Judge McCallin Attorneys 2.6 Judge McCallin Non-Attorneys 3.4 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Overall Overall Attorneys Attorneys Being able to identify and 2.8 3.4 2.7 3.3 78 analyze relevant facts Basing decisions on evidence 2.5 3.2 2.5 N/A 65 and arguments Issuing consistent sentences when the circumstances are 1.9 3.1 1.9 N/A 31 similar Being fair and impartial to both 2.7 3.3 2.7 N/A 66 sides of the case Consistently applying laws and 2.5 3.2 2.5 N/A 65 rules Giving reasons for rulings 3.4 3.5 N/A 3.4 12 Willing to make decision without regard to possible outside 3.5 3.5 N/A 3.5 11 pressure 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 8 ## Page 9 Communications 0 1 2 3 4 Judge McCallin Overall 3 District Judges Overall 3.6 Judge McCallin Attorneys 2.9 Judge McCallin Non-Attorneys 3.8 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Overall Overall Attorneys Attorneys Making sure all participants 3.1 3.6 3 3.7 78 understand the proceedings Providing written communications that are clear, 2.7 3.3 2.7 N/A 62 thorough and well reasoned Using language that everyone 3.7 3.7 N/A 3.7 11 can understand Speaking clearly so everyone in the courtroom can hear what’s 3.8 3.7 N/A 3.8 11 being said 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 9 ## Page 10 Demeanor 0 1 2 3 4 Judge McCallin Score 2.9 District Judges Overall 3.5 Judge McCallin Attorneys 2.8 Judge McCallin Non-Attorneys 3.4 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Score Overall Attorneys Attorneys Giving proceedings a sense of dignity 3 3.5 2.9 3.6 80 Treating participants with respect 3 3.5 2.9 3.4 79 Conducting his/her courtroom in a 2.8 3.4 2.7 3.3 77 neutral manner Having a sense of compassion and human understanding for those who 3.3 3.5 N/A 3.3 11 appear before him/her 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 10 ## Page 11 Diligence 0 1 2 3 4 Judge McCallin Score 2.7 District Judges Overall 3.4 Judge McCallin Attorneys 2.6 Judge McCallin Non-Attorneys 3.5 Judge Judge District Judge McCallin Number of McCallin Judges McCallin Non- Responses Score Overall Attorneys Attorneys Using good judgment in application of relevant law and 2.6 3.2 2.6 N/A 66 rules Doing the necessary “homework” and being prepared 2.8 3.5 2.6 3.5 77 for cases Being willing to handle cases on the docket even when they are 2.6 3.4 2.6 N/A 62 complicated and time consuming Beginning court on time 3.5 3.6 N/A 3.5 11 Maintaining appropriate control 3.5 3.7 N/A 3.5 10 over proceedings Setting reasonable schedules for 3.3 3.6 N/A 3.3 10 cases Managing court proceedings so 3.3 3.5 N/A 3.3 10 that there is little wasted time 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 11 ## Page 12 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 12 ## Page 13 Fairness 0.0 1.0 2.0 3.0 4.0 Judge McCallin Score 3.3 District Judges Overall 3.5 Judge McCallin Attorneys Judge McCallin Non-Attorneys 3.3 Judge Judge Judge District Number of McCallin McCallin Non- McCallin Score Judges Overall Responses Attorneys Attorneys Giving participants an 3.6 3.6 N/A 3.6 12 opportunity to be heard Treating those involved 3.3 3.5 N/A 3.3 12 in the case without bias Treating fairly people who represent 3.4 3.6 N/A 3.4 7 themselves Giving each side enough time to present his or her 3.2 3.5 N/A 3.2 11 case 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 13 ## Page 14 Survey of Appellate Judges For Judge McCallin 12 appellate judges agreed they had worked with Judge McCallin enough to evaluate their performance. 0.0 1.0 2.0 3.0 4.0 Judge McCallin Score 3.7 Judge McCallin District Judges Number of Score Overall Responses Overall performance as a judge 3.7 3.6 12 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 14 ## Page 15 Appendix 1. Survey Methods – Attorney and Non- attorney Methodology and How to Read Results For Judge McCallin, 79 individuals completed surveys with at least a single rating question answered. This report reflects these 79 responses. The survey results are divided into eight sections: Retention, Case Management, Application and Knowledge of Law, Communications, Demeanor, Diligence, Fairness, Strengths, and Weaknesses. a. Response rates Attorneys During the 2018 administration, a total of 16,364 survey invitations were sent to 5,769 attorneys inviting them to evaluate judges and justices receiving interim reports in 2019. On average, each attorney was asked to evaluate 2.8 judges. In total 3,837 surveys were completed with an additional 2,309 responses where the attorney indicated that they did not have enough experience with the judge to be comfortable evaluating him or her. The response rate for the survey was 38% and the survey completion rate (the number of those familiar enough to evaluate the judge divided by the total number of attorney responses including those indicating they did not have sufficient familiarity to evaluate the judge) was 62%. Non-attorneys During the 2018 administration, a total of 143,430 survey invitations were sent to non-attorneys inviting them to evaluate judges and justices receiving interim reports in 2019. In total 4,862 surveys were completed with an additional 1,925 responses where the individual indicated that they did not have enough experience with the judge to be comfortable evaluating him or her. The response rate for the survey was 6% and the survey completion rate (the number of those familiar enough to evaluate the judge divided by the total number of responses including those indicating they did not have sufficient familiarity to evaluate the judge) was 72%. b. Methodology The 2018 attorney survey was conducted in 4 quarterly cycles online beginning on June 4th, 2018. Attorneys with appearances in front of judges during the first quarter of 2018 were first mailed a pre-notification letter on June 4th, 2018 informing them about the survey and providing a link and login information to access the survey online. Next, a series of three email invitations were sent on June 7th, June 18th, and July 2nd, 2018. This process was repeated among attorneys with appearances in the second quarter of 2018 beginning with a pre- notification letter sent on August 10th, 2018. The letter was followed up by email invitations sent on August 13th, August 27th, and September 11th, 2018. For those attorneys with appearances during the 3rd quarter of 2018, the process was repeated again with a letter sent on November 26th and a series of emails sent on November 29th, December 11th, and December 29th, 2018. To further increase the amount of data collected, an additional cycle of data collection took place in January and February 2019. Invitations were emailed to attorneys with appearances during the 4th quarter of 2018. This cycle included a pre-notification letter sent on January 25th followed up by email invitations sent on February 5th, February 11th, and February 19th, 2019. Additional invitations and reminders were sent out on request throughout the data collection process. Data collection among non-attorneys also featured four quarterly cycles with each cycle consisting of a mailed invitation letter with instructions to access the survey online followed by a mailed survey booklet with a prepaid 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 15 ## Page 16 return envelope. To meet tight reporting deadlines, the order was switched for the 4th quarter administration and the survey booklet was sent first followed by the letter. Due to the ability to contact court staff via email, respondents were split into two groups for data collection: court staff and other non-attorneys. The court staff group includes staff members, interpreters and probation officers. The other non-attorney group includes jurors, defendants, witnesses, litigants, and law enforcement personnel. Court staff members were invited via emailed invitations sent on November 5th, 2018 and reminders sent on November 13th and 26th. In addition to the main non-attorney survey where respondents were invited to participate, there was also a citizen feedback survey available to all citizens on the OJPE website. This survey was open for the entire data collection period and data was downloaded for analysis on February 21st, 2019. During this period 87 valid responses were received. The survey remained open and any responses received after February 21stor for judges not receiving an interim evaluation in 2019 were held over for the 2020 evaluation cycle. Table 1: Non-Attorney Data Collection Dates Cycle Invitation Letter Survey Booklet Q1 5/18/18 6/20/18 Q2 8/2/18 9/7/18 Q3 11/9/18 12/19/18 Q4 1/31/19 1/23/19 c. Questions In the core of the survey, attorneys evaluated district and county judges on 17 aspects of judicial performance and appellate judges on 12 aspects of judicial performance using a grade scale of A, B, C, D, or F. These aspects were grouped by topic into different categories, five for district and county judges and two for appellate judges. The district and county categories were: Case Management, Application and Knowledge of Law, Communications, Demeanor, and Diligence. Questions regarding appellate judges were divided into two categories, one for general questions and one specific to their writing (only asked of those who indicated they had experience with the judge or justice’s written opinions). In a final question, respondents were asked if they thought whether the judge met judicial performance standards The question wording for the core of the survey was carried over from the 2017 administration with only minor changes to make the survey gender neutral. The questions were originally developed in 1998 to meet the criteria outlined in statute 13-5.5-101 et seq. Non-attorney respondents evaluated judges on 19 aspects of judicial performance using the same grade scale of A, B, C, D, or Fail. In a final question, respondents were asked if they thought whether the judge met judicial performance standards. The overall structure of the survey was similar to the attorney survey, but the individual rating questions were tailored to aspects that could be rated by those without specific legal experience. d. Analysis and Reporting Letter grades were converted to a numerical score where A = 4, B = 3, C = 2, D = 1 and Fail = 0 for analysis. The results include an overall grade, a grade for each category, as well as a grade for each question. The overall score is calculated by averaging the responses to all questions answered by the attorneys. This score will have the same numerical range as the individual questions from zero to four. Each category score is calculated by averaging the responses to all questions answered by the attorney within each category. This score will have the same zero to four numerical range as the individual questions. Similarly, 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 16 ## Page 17 an average score is calculated for each individual question with the exception of the final question on meeting performance standards. The overall average and category scores will be reported for each judge along with the average scores for the judge’s peers. The average score (with the exception noted above) will also be reported for each question along with the peer group score. In addition, the report will include the distribution of responses for each question, i.e. the percentage of attorneys that assigned a rating of A, B, C, D, and F. The distribution of responses is also reported for the question on retention. e. Comments At the end of each group of questions respondents had the option of leaving comments about the judge’s performance in that area. By statute, these comments are confidential and only provided to the judge and the District Commission on Judicial Performance. They are not released to the public when the rest of the report is released. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 17 ## Page 18 Appendix 2. Survey Methodology - Appellate Judges Methodology and How to Read Results a. Response rates Invitations were sent via email to all 29 Supreme Court justices and Court of Appeals judges. Of these, 12 completed the survey and felt they had sufficient knowledge of Judge McCallin to evaluate their performance. b. Methodology Appellate judges were surveyed to evaluate the performance of district judges. This evaluation of district judges was conducted via an online survey hosted in the Voxco survey software. An email invitation was sent on January 29th, 2019. c. Questions Due to the large number of judges being evaluated, the district judge evaluation survey consisted of a single question pertaining to each judge. Appellate judges and justices were asked to evaluate the district judge’s overall performance as a judge on a grade scale of A-F with A being “Excellent” and F being “Fail”. In the survey, the district judges being evaluated were grouped by district with the districts presented in random order to reduce bias. d. Analysis and Reporting Letter grades were then converted to a numerical score where A = 4, B = 3, C = 2, D = 1 and Fail = 0 for analysis. The overall score is calculated by averaging the responses to all questions answered. This score will have the same numerical range as the individual questions from zero to four. The overall average will be reported for each judge along with the average scores for the judge’s peers. In addition, the report will include the distribution of responses for each question. That is, the percentage of respondents that assigned a rating of A, B, C, D, and F. e. Comments Respondents were given the option to leave supporting comments in a box next to where they graded each judge. By statute, these comments are confidential and only provided to the judge and the District Commission on Judicial Performance. They are not released to the public when the rest of the report is released. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 18 ## Page 19 Appendix 3: Judge Response Counts by Type of Respondent Number of Undeliverable/ Cooperation Respondent Type Total Sent Completes* Responses Not Applicable Rate Attorney 245 119 0 66 55% Law Enforcement 19 0 1 0 - Litigant 557 9 135 6 67% Juror 0 0 0 0 - Others 164 8 17 7 88% Total Non-Attorneys 740 17 153 13 76% Total Respondents 985 136 153 79 58% *Completed surveys include respondents who said that they have sufficient experience to evaluate the judge. 2019 Judicial Performance Survey Report for Judge Andrew P. McCallin 19
    • 2016 Retention Survey Report(PDF on OJPE)
      Structured scores pending

      Full numeric extraction isn’t complete for this cycle’s layout yet. Open the source PDF above for the underlying data.

Score cards render where the PDF extractor found numeric fields (partial is shown with “—” for missing tiles). “Read full survey text” surfaces the complete report transcription for every survey on file. Source PDFs on OJPE are always linked above for fidelity. Coverage note on Sources.

Sources