URGENT: Departmental Evaluation and Chair Survey

Dear Deans and Chairs,

As some of you may have very recently heard, we have been tasked by our Board of Regents to compile data on our academic departments for a Board Retreat being held October 19th. The Board of Regents have specifically requested that all academic departments be assigned an overall ranking of “high”, “medium”, or ”low” based on effectiveness and productivity.  Importantly, there were no metrics or methodologies suggested, nor is there a sense of how many departments should fall into each category.  Equally importantly, I don’t believe the Regents are interested in “reaching in” to Academic Affairs in a proscriptive fashion.  However, they do want to see immediate movement towards the goal we all share – namely, becoming a Tier 1 research university. 

To aid in this process, I will need data input from Deans/Chairs by Tuesday, September 30 (see below).  While it was my intention to begin a thoughtful, iterative and transparent process for departmental evaluation this academic year, I had not anticipated that such a process would need to be initiated so early in the semester, and without time for development of an agreed upon process, data verification, etc..  I fully appreciate and anticipate the concerns that you may have with these aggressive timelines.  But the task before us is clear and urgent, and it does set us on a pathway to a much better university.

This first pass at departmental evaluation will be what I am viewing as “Phase I”, which will show our ability and determination to actually evaluate all UNT departments concurrently – a process that has not been carried out at UNT in more than 20 years, to my knowledge.   Phase 1 will, given the schedule, be based on the less-than-perfect data that we can gather in this short period of time.  Additionally, it will compare UNT departments with other UNT departments, which is less useful than external benchmarking.  Moreover, the metrics to be employed in Phase 1 will not be weighted (hastily determined weightings are worse than no weightings).  As such, I acknowledge that this first Phase will be imperfect, but it will fulfil the purpose of showing that we are engaged in a quantitative process of department evaluation.  The ultimate goal (Phase 2) will be to compare each UNT department to similar or identical departments around the nation, based on data that are as accurate as possible and metrics that are weighted.      

The process we will use for evaluation in Phase 1 will leverage our use of existing institutional programming most frequently associated with the THECB Academic Program Review process, as well as data definitions and practices furnished under the Texas Administrative Code.  A complete list of definitions and methodology related to the 18 Characteristic data is available.  This evaluative tool was endorsed by the Faculty Senate in 2011, and is currently used as part of our Graduate Program Review process. 

I have tasked our Office of Institutional Research and Effectiveness to make this process as efficient and easy as possible for our Chairs, by populating as much of the evaluation rubric as possible from central sources. IR&E staff will compile information on the following variables (following the data definitions described in the link above):

  • Number of Degrees Awarded per Year
  • Graduation Rate by Cohort Year
  • Time to Degree by Cohort Year
  • Admission Criteria – Average SAT Score
  • Percentage of Full-Time Graduate Students with Financial Support
  • Average Financial Support Provided (Monetary Value)
  • Core Faculty to Student (FTE) Ratio
  • Core Faculty External Grants
  • Percentage of Full-Time (FTE) to Headcount
  • Number of Core Full-Time Faculty
  • Student Diversity – % Minority
  • SCH Generated
  • Student Satisfaction Survey (4 Questions from Graduation Survey)
  • Cost per course by Instructor Rank
  • Program Facilities and Equipment- Gross Square Feet
  • Program Finances and Resources – Allocable Budget

    However, we will need some important data that only Chairs and/or Deans can provide. Each Department will need to provide information regarding:

  • graduate student employment status after graduation,
  • a summary number of faculty scholarship for the past calendar year,
  • the value of the departments recruiting scholarship and
  • the department’s program goals.  

We have created a Qualtrics survey to aid in the data gathering process and to ensure consistency across departments. Once all of the data points have been collected, IR&E will convert these to Z-scores.  Z-scores are very useful for combining and ranking scores from performance data that has different means, ranges, and standard deviations.  We will then create a summed up Z-score that will allow us to look at outliers (both high and low performers) that differ from the norm. Your Dean is actively involved in this process and helping to review and validate these data.

I want to emphasize that a department identified as a low performer could be there for several reasons including    recent loss of faculty or poor facilities.  However, a low performing department may also have relatively few students who consequently are educated at very high per student cost by a faculty with low scholarly productivity.  As you can imagine, the interesting part of the process will be determining which departments fit into which categories (Phase 1) and then why they may be there and what is to be done about it (Phase 2)!

Chairs, please complete your survey input by end of day Tuesday, September 30, 2014.  The survey can be accessed by visiting:  http://tinyurl.com/untdeptsurvey

Once we receive the data from Chairs, we will do as much data validation as can, and be as transparent and interactive as time permits.  At the same time, we are under a very compressed timeline as we must deliver back to the Board a completed analysis for the October 19th deadline.  For Academic Affairs to create this deliverable we will need the support of every one of our departments.

If you have questions please do not hesitate to contact Allen Clark or me.  Thanks for your help with this important task!