Scoring Rubric for Baylis/Burwell VMI. Image courtesy of Presenter Media
This post is a continuation of my previous post [KPI – Part IV: Guiding Principles]. It will describe the scoring rubric we selected to use to assign points to institutions on the Guiding Principles Factor of the Baylis/Burwell Vitality/Morbidity Model.
Two different approaches to building our scoring rubric. Image courtesy of Presenter Media
There are two different directions which we could have taken to develop our scoring rubric. The first way was an ultra-quantitative, spreadsheet approach attempting to measure the quality of the statements of institutional Mission, Vision, and Core Values, and the institution’s efforts to live out those statements in their actions.
This type of approach is typically called the objective approach. However, if by objective you mean “not influenced by personal feelings or opinions in considering and representing facts,” this direction is far from being truly objective. There are many points of subjectivity present in the quantitative scoring of the various components and in the weighting factors used in combining component scores to obtain a final score, where the raters’ biases and opinions enter into the equation.
A panel of higher education experts weigh the evidence and make judgments on each aspect of a particular factor. Image courtesy of Presenter Media.
The second approach is a more holistic approach which is typically labeled as a subjective approach. It relies on the use of higher education experts, who have had years of training and experience in the field of higher education, to evaluate the institution in a number of ways.
The first thing these experts are asked to do is to read the institution’s published documents and judge whether they believe the institution has selected values and behaviors that represent those of a quality institution of higher education. The institutions are scored on the following three-point scale:
Is the institution a stellar citizen of the higher education community or a devil in disguise? Image courtesy of Presenter Media
-1 Totally inadequate for a quality institution of higher education
0 Barely adequate for a quality institution of higher education
+1 Describes a high performing institution of higher education
The higher education experts are then asked to judge whether the behavior of a given institution matches its stated beliefs using the following scale:
In the opinion of the higher education experts does the institution’s behavior match its stated values. They will weigh the evidence and make their decision on their training and experience in higher education. Image courtesy of Presenter Media
-1 Behavior doesn’t come close to its stated values. The institution fails to meet its own stated standards
0 Behavior barely meets its stated values or standards.
+1 Behavior exceeds the expectations set by its stated values.
A quality institution of higher education should be beyond reproach. In light of this, the panel of higher education experts is asked to make two more judgments. The first judgment involves the institution’s track record with those entities and agencies to which the institution is responsible. Does the institution meet all of its required reporting deadlines and fulfill all obligations to federal and accrediting agencies? Institutions will be scored on the following scale:
The institution has done everything it could to move it to the top of its class. Image courtesy of Presenter Media
-1 The institution has failed to meet more than one reporting obligation or legal requirements.
0 The institution has met all requirements and obligations but has occasionally been late or hesitant in making results public.
+1 The institution has gone of out its way in meeting requirements and obligations. It has made been completely transparent in all of its operations.
The final area of concern for the panel of experts deals with the reputation of the institution. The panel will judge whether the institution is held in high esteem by various entities such as higher education as a whole, the general public, students and prospective students, and employers of the institution’s graduates.
Is the institution a stellar citizen of the higher education community or a devil in disguise? Image courtesy of Presenter Media
The scoring scale for this area of concern is as follows:
-1 The reputation of the institution is tarnished in a number of areas with a number of groups.
0 The reputation of the institution is considered “run-of-the-mill.” It is not outstanding in any area.
+1 The reputation of the institution is stellar with all groups with which it deals.
To determine a factor score for Guiding Principles, the sub-factor scores are summed. Total scores are assigned as follows:
If the total sub-factor score is -3 or less, the assigned factor score is -1. Any institution in this area should be considered in trouble and possibly dying.
If the total sub-factor score is -2 to +2, the assigned factor score is 0. An institution with a score in this area is just hanging on and should be considered just surviving.
In institution in this category is considered a top-tier or elite institution. It is truly thriving. Image courtesy of Presenter Media.
If the total sub-factor score is +3 or more, the assigned factor score is +1. An institution with a score in this area is doing well and should be considered to be thriving.
With the institutions we have examined we have found a predisposition away from the thriving side of the scale. It should not be surprising. Most observers will readily say that the overwhelming majority of colleges and universities are either in trouble or just surviving. There are few elite, or top tier institutions that are really thriving.
Next Tuesday, March 19, I will take a break from this series of post on Key Performance Indicators and publish a special post inspired by the scores of birthday wishes that I received this past week. It may be unusual to throw a big celebration for someone’s 73rd birthday. However, after a series of traumatic brain incidents more than a decade ago, scores of doctors wouldn’t have given you a plug nickel that I would make my 73rd birthday. Thus I will publish a post celebrating an unexpected decade of extra life. What would you do with an extra decade of life?
Inactivity, inattentiveness, and other bad business practices lead to the failure of any organization. Image courtesy of Presenter Media.
As noted in the previous post, Key Performance Indicators – Part II: Definition, the theme of this post was going to be A University Should Be Managed as If It Were a Business. In all of my previous roles, as a university administrator or the creator of this blog, I made no efforts to hide my sentiments concerning this proposition. It was always one of my operating premises.
When the wheels fall off an organization, it will fail to run. Image courtesy of Presenter Media.
In 2016, I started a series of posts on the theme The Business Model of All Higher Education Is Broken. Even in the title of the series, I attempted to make the point that institutions of higher education (IHEs) must view themselves as business enterprises. As an academician, I believe that institutions of higher education must be more than businesses. However, if they don’t operate using the best business and management techniques then they will surely fail, which is what we have seen with 2,000 American IHEs since 1950.
Too many universities live in a fantasy world chasing rainbows, leprechauns, and illusory pots of gold. Image courtesy of Presenter Media.
Milton Greenberg in his seminal essay “The University Is Not a Business (and Other Fantasies)” published in EDUCAUSE Review, vol. 39, no. 2 (March/April 2004), argues forcefully that a university should be managed as if it were a business.
Very early in his essay, Greenberg proclaims, “ Presumably, a “business” involves the hierarchical and orderly management of people, property, productivity, and finance for profit.” The primary counterarguments of academicians to Greenberg’s position hinge on three concepts he introduces in this sentence: hierarchical management, productivity, and profit. In three future posts in this series, I will separately tackle each of these counterarguments.
Eight Factor Model of Institutional Vitality developed by By Baylis and Ron Burwell. Image copyright by Higher Ed By Baylis, LLC. Image courtesy of By Baylis and Ron Burwell. Constructed using ClickCharts Software
But first I return to present my argument on why universities should be run more like businesses. In studying the 2,000 deceased IHEs, Ron Burwell and I noticed eight factors that we believe contributed negatively to their vitality and their eventual morbidity. The eight factors are shown in the diagram to the left.
Although the eight factors are obviously not completely independent of each other, they are sufficiently different to warrant separate consideration. Additionally, that consideration would take up too much space for one blog post. Thus, I will address each of the factors in upcoming posts.
Mind Map of the Guiding Principles Factor. Image courtesy of authors By Baylis and Ron Burwell. Constructed using ClickCharts Software.
To give you a taste of how I will be introducing and treating these factors, I present a Mind Map Diagram on the right illustrating the three components which define the Guiding Principles Factor.
Under each of the three components, the diagram presents the major ingredients that go into measuring the success of the organization in that component.
A reasonably informed person weighing the evidence should be able to make an informed judgment. Image courtesy of Presenter Media.
In order to simplify our study, without losing the crux of discovering the reasons why institutions failed, we have chosen to use the straight-forward three-point scale of Thriving, Surviving, and Dying. Instead of attempting to construct complicated, quantitative scales to measure each subfactor of our eight factors, we are going to use a subjective approach similar to Supreme Court Justice Potter Stewart’s take on defining pornography: “I may not be able to define it. But I know it when I see it.”
Vitality/Morbidity Index (VMI) Gauge indicating an institution is greatly struggling in the Guiding Principles Factor. Image courtesy of Presenter Media
With each component in our factors, most reasonable observers can easily determine whether: 1) an organization is extremely successful and thriving in terms of this component, 2) just barely getting by or only surviving, or 3) failing badly and falling far short of success or flat out dying. This approach permits us to use a simple gauge to illustrate the vitality/morbidity level of an institution.
We will also associate a three-point numerical scale with our three categories: Thriving (+1); Surviving (0); Dying (-1). We then added the scores across all eight factors. Repeating this process for each institution in our database of closed colleges and universities, we were not at all surprised to find that the total score of each closed institution was negative. No closed college had a total positive score. Some individual factor scores were positive but they were outweighed by a much larger share of factors with negative scores.
If this model is to have predictive capabilities it must also work with all types of institutions. We have tried our model out on a number of institutions that we identified as thriving, surviving, and outright struggling.
Is the wrecking ball set to knock down your institution? Image courtesy of Presenter Media.
In this process, we did find a number of colleges that were still operating which had negative VMIs. In each of these cases, the colleges involved could easily be classified as struggling or just barely surviving. They were definitely not thriving.
Although I believe that it is difficult to “kill a college” it is not impossible. Just ask the constituencies of Newbury College (MA), College of New Rochelle (NY), Green Mountain College (VT), and Hampshire College (VT).
For institutions that we identified as thriving, just as we expected each of them had a total VMI that was positive. What about the struggling institutions with positive VMIs? We believe that these institutions must address the factors that are negative or “zero” or they could be heading for more serious trouble.
In the post above I outline several different directions that I could go with my next post. At this point, I am working on a post that delves more deeply into the VMI Factor Guiding Principles which I introduced in this post. Watch for it next Tuesday, March 12, 2019.
With each new attempt to open up the topic of Key Performance Indicators, I hit a brick wall. Image courtesy of Presenter Media.
In my post Getting Back in the Saddle from Tuesday, January 21, I indicated that my next post would be about a proposal for the introduction of the Admissions Multiplier Effect, a new Key Performance Indicator (KPI) for higher education. On multiple attempts to compose such a post, I found myself running into a brick wall.
The brick wall was more like a series of hurdles I kept tripping over. Image courtesy of Presenter Media
However, on closer examination, the brick wall turned out to be more like a series of hurdles over which I kept tripping.
The Baylis problem-solving methodology focuses on three questions. Image courtesy of Presenter Media
As a mathematician by training, during my college days I developed my own approach to problem-solving. My approach was centered around three questions. These three questions formed a basic methodological approach which I used in every mathematics, statistics, or science class I ever taught. My three questions were the contextual foundation in each of the three statistics textbooks I coauthored.
Baylis Problem-Solving Methodology involving three questions. Image courtesy of Presenter Media.
The first question and beginning point is “What do I know?” Once that has been established, the second question is “What do I want to know?”
Map out the best path from Point A to Point B. Image courtesy of Presenter Media
If I know where I am and where I want to go, I can then plot out a course that answers the third question “How do I get from what I know to my destination of what I want to know?”
Most of the blood, sweat, and tears are shed during the hours of training. Once you’ve done the heavy lifting, it becomes easier to do it a second time. Image courtesy of Presenter Media.
Faced with any problem, if you can answer these three questions, you have reached the crux of this problem. You have completed the most difficult task in solving this problem. You’re done with the majority of the heavy lifting. You are generally finished with the shedding of the blood, sweat, and tears. The hard work is almost complete. The only thing left to do is to follow the path that you laid out.
I have used this methodology to solve problems not only in all aspects of my personal life, but I have also used this approach in every mathematics, statistics or science course I ever taught. I included these questions in each course syllabi I distributed to students. I encouraged my students to test the effectiveness of the method in their encounters with problems.
In addition, these questions formed the contextual foundation of the three statistical textbooks I coauthored. I view statistics as a problem-solving tool. We included these questions in the opening paragraphs of the Preface of each textbook. Throughout the textbook, the three questions were the outline we used to present our version of a standard “Statistical Method” of problem-solving. We continued to reemphasize them as we introduced each statistical tests. The statistical tools such as ANOVA, chi-square, correlation, F-test, MANOVA, path analysis, Pearson-r, regression, Spearman-rho, t-test, and z-test, become parts of the path to be taken from what one knows to what one wants to know.
Returning to the original problem proposed by this post the introduction of a new KPI for higher education raises a number of other
questions, such as:
How do we know if a college is meeting its operational and strategic goals? Image courtesy of Presenter Media
What is a performance indicator (PI)?
What is a key performance indicator (KPI)?
What are the standard KPIs for higher education?
Many in higher education reject the idea that higher education is a business. Image courtesy of Presenter Media.
Why do so many in higher education believe that higher education is not a business?
Why do so many academicians shun the use of performance indicators in higher education?
Why do I strongly believe the enterprise of higher education should be managed as if it were a business?
What are the admission and retention processes used by colleges? How can we tell if a college is meeting its admissions, retention and graduation goals successfully? Image courtesy of Presenter Media.
What are the primary aspects of the admission and retention processes in higher education?
What are some of the KPIs that are related to the admission and retention processes in higher education?
What is my definition of the Admission Multiplier Effect (AME)?
Why do we need to introduce the AME?
How should we introduce and use the AME?
The figurative brick wall I hit in the first paragraph of this post was the magnitude of work necessary to provide reasonable and intelligible answers to the three questions of the Baylis Problem-Solving Method.
To answer all these questions in one post on which I have imposed a 1000-word limit is an impossibility. In fact, I am not sure how many posts it will take me to answer all these questions. The best thing that I can do is to dive into the pool and start writing.
Since I am fast approaching that self-imposed limit of 1000 words for each post, I will close out this post. In my next post, I will address the question of “What are performance indicators?” Since I haven’t written the upcoming posts yet, at the end of each upcoming post, I will indicate the topic of the next post. Since I am also still trying to address some health issues related to a chronic-fatigue condition, I can’t guarantee my continued ability to maintain the substantial writing schedule that I have laid out for myself. For this reason, I have asked some friends and colleagues to step in and publish a guest post now and then. If any of my readers would be interested in taking a shot at writing such a post, please contact me and we’ll talk.
Finally, back in the saddle and at least writing something. Image courtesy of Presenter Media
It’s been so long since I’ve posted something. Even though I am still fighting medical problems, I’ve been overrun with that urge to get back in the saddle and start writing again.
Reading today’s (1/21/19) Harvard Business Review Management Tip of the Day, I had a thought. Share this on By’s Musings to let people know that I’m still alive and thinking.
How to Work with Someone Who Bugs You
Some people have a way of getting on your nerves. They are as annoying as a blaring trumpet. Image courtesy of Presenter Image
Sometimes you have to work with a colleague you don’t particularly like. They may not be toxic or difficult — they might just get on your nerves. To work with them productively, remind yourself that while you won’t get along with everyone, there is potential value in every interaction. Think about the other person’s point of view: Why do they do the things that annoy you? What might be motivating them? And how do you seem to them? It also helps to approach conversations with a problem-solving mindset:
Working together and sharing thoughts can put the puzzle together quicker. Image courtesy of Presenter Media
“I don’t feel like we are working together as effectively as we could. What do you think? Do you have any ideas for how we can work together better?” If that doesn’t work, try asking for their help: “You’ve been around here longer than I have. What should I be doing more or less of?” This can ease tensions and reboot a difficult relationship because it shows that you value the person’s experience.
Admissions Multiplier Effect – a new proposed KPI for colleges and universities. Image courtesy of Presenter Media.
Stayed tuned to By’s Musings because next Tuesday, January 28, I will be publishing a post on my proposal for a new higher education Key Performance Indicator: the Admissions Multiplier Effect.
Hopefully, I will also be able to outline so what is ahead for me, the projects I’m planning for Higher Ed By Baylis, LLC, and the future posts in this blog.
This post has been very difficult to write and has required multiple drafts. Image courtesy of Presenter Media.
American Higher Education (AHE) has lost its lodestar. This post has been a lot harder to write than I thought it would be. At first, I was extremely excited to write this post. However, as I began to compose it, I immediately hit a number of roadblocks.
The first roadblock related to the way I process thoughts and ideas. As many of you know, almost a decade ago two traumatic brain incidents drastically changed my life. After a burst aneurysm caused the implosion of a benign meningioma attached to my right temporal lobe, I started having trouble finding words. I found myself fighting a case of oral aphasia.
Nine months later, I experienced four tonic-clonic seizures within a thirty-minute timespan. When I awoke from a four-day coma, I found myself no longer thinking in terms of words. I was now processing thoughts and ideas in terms of pictures. Words became my second language. Visual images were now my first language. This meant that in order to communicate with people, I had to translate back and forth between pictures and words.
My mind was overflowing with pictures that spoke to this issue of American higher education. However, translating those visuals into words was much more difficult and slower than usual. If I was able to compose other posts, why had this particular post become so problematic?
During my struggles with this post, I had a eureka moment. In the deep recesses of my mind, I found a perfect word, LODESTAR, that covered four aspects of the topic I wanted to address.
First of all, a lodestar is a fixed point of reference, which can be used to describe the position or motion of one object relative to another object.
There have been dramatic changes in both American higher education and society during the past century. AHE needs a lodestar to position itself in relation to American society.
Secondly, as a fixed point, a lodestar can be used as the foundation of a guidance system to help individuals get from one point to another, particularly when they are lost. AHE has lost its way. It needs a GPS.
It is wandering aimlessly from one crisis to the next. Headline after headline decries its loss of effectiveness in serving the needs of American society, its declining support in public circles, and its strident and stubborn insularity.
In article after article, questions are raised about the declining confidence of American society in higher education, and the seeming indifference of AHE to the external demands for change. The internal conflicts among the primary actors within AHE are laid bare to the public, exposing all to criticism and contempt.
Comparisons between education for life and career education are plentiful. The philosophical and theoretical bases for liberal and professional education are made public for everyone to pick a side.
In analysis after analysis critics and proponents explore hypotheses about the rising cost of higher education, the short and long-term effects of the staggering debt load that students and institutions are accumulating, the commercialization of higher education, and the adjunctification of the faculty.
Thirdly, lodestars are models of propriety. They live by fixed values and principles, no matter what the cost to them or their institutions. Currently, it seems that every week brings another scandal to light in American higher education. No segment of AHE has escaped unscathed.
Finally, a lodestar is an inspirational leader. I challenge you to name one individual in educational circles today who inspires others to follow him or her.
Individual campuses may have local lodestars. However, where are the likes of Ernie Boyer, Lee Shulman, John Dewey, Mortimer Adler, Maria Montessori, Paulo Freire, Bertrand Russell, Maxine Greene, B. F. Skinner, James Conant, and Martin Buber? At this junction of time, the enterprise of American higher education has no one individual who stands out ahead of the rest of the field.
As I was pulling my thoughts together for this post, a political/media circus in America made the term lodestar a laughing matter and a joke. How could I seriously use it in this post about American higher education? I decided that I could use it, and must use it because it is the right word to use.
Having settled on the inclusion of the word lodestar, there were still two major stumbling blocks with respect to this post. The first related to the format I have used in all my recent posts. After I translated the pictures in my head into words, I took another step. Since I am not an artist and can’t draw an intelligible sketch of anything, I went and found free pictures that would duplicate as closely as possible the visions in my head. I did this to help my readers understand my thought processes.
In this case, I drew a complete blank. I found nothing that came close to communicating my thoughts. Thus, I have no visual robes to wrap around my verbal thoughts. I decided to go ahead and present the unadorned thoughts to my readers. I do have one question for my readers: Which style do you prefer? The verbal thoughts augmented with pictures, or the naked thoughts by themselves? Please tell me in the comment in the box below. I will use this rough survey to help me determine how I will proceed with my future posts. The second stumbling block was the 1,000-word limit. This will require multiple posts on this topic which will follow in future weeks.
In my next post, scheduled for publication on Tuesday, October 23, I return to my roots as a mathematician and an institutional researcher. I will introduce a new Key Performance Indicator (KPI) that I developed. I call it the Admissions Multiplier Effect. I believe it provides important information that is not otherwise available and should be featured on the dashboard of every institution of higher education.
How should we improve or fix a broken structure? All images in this post are courtesy of Presenter Media.
This is the initial post in my Point versus Counterpoint thread. The proposal that I wish to address is the following: “When faced with the profound challenge of making significant changes to an existing program, facility or policy, what is the best approach for an institution to take?” Should the organization remodel the existing structure, or tear it down and completely rebuild a new structure from the ground up?
What happens when the pieces don’t fit together just right? You get a lot of pushing back and forth.
I have seen battles over this question severely divide more than one campus. Many times within education, these battles degenerate into classic clashes between traditionalists and disrupters, between evolutionists and innovators, or between the old guard and the young Turks.
In the quintessential debate approach of Point versus Counterpoint, it would be incumbent upon me to select a side on the “Repurpose or Build Anew” question. During my 50 years in the academy, I have been known as a traditionalist who studied and revered the best aspects of education’s rich history.
During 35+ years as a college administrator, I also had a reputation as being an approachable leader who listened carefully and made thoughtful decisions based upon all the evidence. These two characteristics might suggest that I should assume a role as a supporter of the “repurpose” side.
However, throughout my career, I have been acknowledged as an educational entrepreneur. I have been recognized for my ability to think outside the box while still accommodating those inside the box. Often I championed new and different approaches to problem-solving when the old methods were not working. I have been known for pushing for innovation and change when change is needed.
On the Rogers Adoption/Innovation Curve most of my former colleagues would place me in the Innovator or Early Adopter segments. I have always been known as someone who was eager to find new solutions to long-standing problems and pushed the limits on how the technology could help. These characteristics would suggest that I should assume the role of a supporter of the “build anew” side.
Even though I have had a 60+ year love affair with education, I am deeply concerned about its future. Given my recent work on the financial models of education and my research into the demise of more than 1600 American colleges or campuses since 1950, I see so much that is broken in American higher education that I often wondered where it is heading.
Since this is my blog, I will take an owner’s prerogative and assume the compromise position of favoring “Building Anew, Except in Very Limited Cases, When Repurposing Is Appropriate and the Most Feasible Approach.”
Why do I believe that “building anew” is the best choice for American higher education? Let me count the ways that I believe American higher education is in trouble.
American higher education has lost its lodestar. Where is the inspirational, values-based, principled leadership that developed the most advanced, highest quality system of higher education in the world?
The three segments of American higher education (public, non-profit private, and proprietary) treat each other as enemies and competitors rather than allies.
The basic financial model of American higher education is broken. How can a system survive that relies on billions of dollars annually from endowment and donors, and complains when those donors ask for something in return? How can a system take billions of dollars from public coffers and then balk at questions of accountability? How can a principled-system saddle its consumers (students) with more than $1.3 Trillion in debt load?
The internal structure of most institutions of higher education in American consists of isolated silos which have little to no communication with each other. Within most colleges, the right hand has no idea what the left hand is doing.
American higher education has seemingly pushed the individuals who should be the most important persons in the system, the students, to the periphery. Investors are only interested in their Return on Investment (ROI). Administrators and faculty bicker constantly, bitterly accusing each other of sabotaging the enterprise and only looking out for their own self-interests. Students and parents complain incessantly that no one is listening to them.
Many students, parents, and politicians act as if education is an entitlement rather than a labor-intensive, responsibility. Debates on whether students should be given the rewards of education without the expending the hard work to earn them are waged privately across campuses and publicly in the media.
American education has fallen into the trap of the “Procrustean Bed” thinking one form of education fits all students and one measuring stick is sufficient for all institutions.
Society rallies around the banner of American higher education raised as the clarion call for social mobility. Community leaders then throw their hands up in despair when the data show it is not working. They conveniently forget that history suggests and the data show that education institutions tend to be excellent reflections of our society and not particularly effective change agents. Yes, there are individual victories. However, there have been too few to change our society as a whole.
Do you have a piece of the puzzle that I have missed. Please let me know what it is.
Readers, it is now your turn to engage in this conversation. Are there problem areas that I have missed? Please let me know now. In future posts, I intend to individually address each of the above areas. Readers, if you have a different take on those areas, you will an opportunity to weigh in on those areas at that time.
My next post, scheduled for Tuesday, October 16, will begin to address the issue of the lost lodestar of American higher education. Thank you for joining this journey. Enjoy your coffee and the conversation.
