Forbes College Ranking: True Gauge or Sketchy Data?
Forbes recently released their annual rankings of colleges. The rankings were calculated in partnership with the Center for College Affordability and Productivity. My institution was conspicuously absent, and I was asked to investigate why. (The responsibility for responding to external credibility surveys lies within my unit.) What I have subsequently found is that the rankings are based on existing, publicly available data (IE no surveys were sent to each institution requesting data, information.) Which in and of itself would not trouble me, except that the data points used, to me and others, are questionable at best. Additionally, information like the infamous “other cost” category institutions report as an allowable cost at the discretion of the student to cover expenses like mileage to clinicals, internships, etc. in their financial aid package, are treated as actual billed charges, therefore increasing the “cost of attendance” and subsequently this elusive “net price” calculation and the debt load calculation (predicted, I might add).
I have several excerpts from articles, blog posts, and even their own methodology posted below giving a glimpse of the, in my humble opinion, “sketchiness” of this ranking. I can only hope that Joe and Jane Sixpack are able to sort through the variables…oh wait, they likely can’t. So now we have another “think tank” with a clear agenda (political or otherwise) leveraging a brand like Forbes to advance their cause.
How important are rankings like this and U.S. News and World Report Best Colleges? Only the audiences we are trying to attract will tell. And believe me, I intend to ask them just that, so we can tailor our approach accordingly to these surveys.
Compiling the Forbes /CCAP Rankings (excerpt from the methodology document, full document can be found on their site)
By the Staff of the Center for College Affordability and Productivity
Ranking Factors and Weights
The Center for College Affordability and Productivity (CCAP), in conjunction with Forbes , compiled its college rankings using five general categories, with several components within each general category. The weightings are listed in parentheses:
1. Student Satisfaction (27.5%)
- Student Evaluations from RateMyProfessor.com (17.5%)
- Actual Freshman-to-Sophomore Retention Rates (5%)
- Predicted vs. Actual Freshman-to-Sophomore Retention Rates (5%)
2. Post-Graduate Success (30%)
- Listings of Alumni in Who’s Who in America (10%)
- Salary of Alumni from Payscale.com (15%)
- Alumni in Forbes/CCAP Corporate Officers List (5%)
3. Student Debt (17.5%)
- Average Federal Student Loan Debt Load (10%)
- Student Loan Default Rates (5%)
- Predicted vs. Actual Percent of Students Taking Federal Loans (2.5%)
4. Four-year Graduation Rate (17.5%)
- Actual Four-year Graduation Rate (8.75%)
- Predicted vs. Actual Four-year Graduation Rate(8.75%)
5. Competitive Awards (7.5%)
- Student Nationally Competitive Awards (7.5%)
School Selection
The 650 institutions of higher education in this ranking are schools which award undergraduate degrees or certificates requiring ―4 or more years‖ of study, according to the U.S. Department of Education, and only those schools categorized by The Carnegie Foundation as Doctorate-granting Universities, Master‘s Colleges and Universities, or Baccalaureate Colleges are included in this sample of schools.
Of the 650 schools included in the sample, 608 wereincluded in the 2010 college ranking. (A total of 610 schools were ranked in 2010, but two of them, Bryant University and Missouri University of Science and Technology are now classified as ―Special Focus‖ institutions by the Carnegie Foundation). We have accounted for any name changes that have occurred over the past year. The 42 schools added this year to the sample are all institutions classified by the Carnegie Foundation as Doctoral/Research Universities and were added based upon undergraduate enrollment size.
A Little History of the Forbes Rankings from 2008-present, excerpt from a commentary on methodology (full commentary can be found at: http://bestcollegerankings.org/popular-rankings/forbes-college-rankings/)
2008 marked the first year that Forbes entered the college ranking fray. They choose to use a methodology that included the following percentages: Listing of Alumni in the 2008 Who’s Who in America (25 percent); student evaluations of professors from Ratemyprofessors.com (25 percent); four-year graduation rates (16 2/3 percent); enrollment-adjusted numbers of students and faculty receiving nationally competitive awards (16 2/3 percent); average four year accumulated student debt of those borrowing money (16 2/3 percent). They did not break colleges down into different schools as U.S. News does, but instead choose to separate private and public colleges instead.
Methodology: In conjunction with Dr. Richard Vedder, an economist at Ohio University, and the Center for College Affordability and Productivity (CCAP), Forbes inaugurated its first ranking of America’s Best Colleges in 2008. They based 25 percent of their rankings on seven million student evaluations of courses and instructors, as recorded on the Web site RateMyProfessors.com. Another 25 percent depended upon how many of the school’s alumni, adjusted for enrollment, are listed among the notable people in Who’s Who in America. The other half of the ranking was based equally on three factors: the average amount of student debt at graduation held by those who borrowed; the percentage of students graduating in four years; and the number of students or faculty, adjusted for enrollment, who have won nationally competitive awards like Rhodes Scholarships or Nobel Prizes. CCAP ranked only the top 15 percent or so of all undergraduate institutions.
Negative Commentary on the Methodology (Excerpt from Suite101.com: The Forbes Best College Rankings 2011: Are They Kidding?)
What Goes in Must Come Out
First of all, a quick review of the Forbes methodology. It is the goal of the rankings to evaluate college as a consumer or investor would evaluate a commercial product. The focus is on the return on investment–for what you pay, do you get a good “value”? The most important element in assessing this value is “Post-Graduate Success,” accounting for 30 percent of the total.
This “success” is measured by the salaries of graduates as reported by Payscale.com; membership in “Who’s Who”; and by alumni representation on a list of corporate officers chosen by Forbes and the Center for College Affordability and Productivity (CCAP). CEOs and board members of leading companies are the only persons who are eligible, thereby narrowing the definition of “success” to achievement in the business world only.
It is interesting that Forbes would allow use of “Who’s Who” listings as a measure of college success. In a 1999 article for the magazine called “The Hall of Lame,” Tucker Carlson, a Fox News commentator, derisively showed how inclusion in Who’s Who publications did not require notable achievement.
Another 17.5 percent of the total is based on student evaluations of instructors, taken from the website Ratemyprofessors.com. While student evaluations are useful, they can also lead professors to emphasize popularity at the expense of scholastic rigor.
An additional 17.5 percent of the total comes from actual and anticipated four-year graduation rates. Using four-year rates rather than six-year rates clearly favors colleges that are wealthy enough to subsidize virtually all eligible students based on need or merit, or whose student body is made up of highly-prepared students with sufficient economic support. State universities, whose students often have to work part-time or even take a semester off from school, usually cannot match the four-year graduation rates of private colleges.
Likewise, the rankings penalize colleges whose students have higher student debt loads, and this also slants the rankings toward wealthy colleges and parents.
Academic Reputation—Forget It
The most glaring deficiency of the Forbes survey is that the only standard it uses to assess the intellectual credibility of a college is the data from Ratemyprofessor.com. Academic reputation and faculty achievement count for nothing, even though a recent UCLA study of more than 200,000 freshman across the country revealed that undergraduate academic reputation was the most important factor for these students when they were choosing a college. Forbes wants to change that perception, but does the magazine really believe that reputation counts for nothing in the business world as well?
It is ironic that a survey that is supposed to be student-centered disregards the one factor that students themselves cite as being most important to them: quality. Interestingly, the UCLA study also showed that prospective students are learning to be guarded in their use of college rankings, a healthy sign indeed.
“In God We Trust, All Others Must Bring Data”
I love the quote from W. Edwards Fleming, “In God We Trust, All Others Must Bring Data”. [Excerpt from Wikipedia: William Edwards Fleming (October 14, 1900 – December 20, 1993) was an American statistician, professor, author, lecturer, and consultant. He is perhaps best known for his work in Japan. There, from 1950 onward, he taught top management how to improve design (and thus service), product quality, testing and sales (the last through global markets) through various methods, including the application of statistical methods.]
I came across a few blog posts/articles recently addressing marketing trends, stats, etc., that I found interesting. Just sharing light reading to consider as we continue our marketing and advertising efforts!
- 100 Awesome Marketing Stats, Charts, Graphs, Data
- The Demographics of Social Media
- The Growth of Mobile Marketing and Tagging
How are we paying attention to data? How do we have the right blend of data versus instinct? For my job specifically, how is higher education marketing and recruiting adapting to the changing consumer behavior, specifically on the web, and getting away from the direct mail (spray and pray, shotgun method, etc.) strategy? What is a good percentage of your budget to move away from the “tried and true” methods and test in some of the strategies described in the following posts? So much to consider!
University Branding Takes to the Air
I know this is not “new” news, but the recent addition to the fleet of Horizon Airlines airplanes to don public university branding caught my attention again. The Montana State University Bobcat theme brings the total to eight planes in the airline’s fleet promoting public universities in the northwest.
How do you measure success of that brand advertising initiative?? More importantly, how do you justify the cost? I have no clue what the price tag was for those branded planes, nor am I aware of the budget situation in these states. I do however know about the state of Illinois budget situation and the resulting pinch the public universities are facing. (And pinch is an understatement!) I would find it very hard to justify the cost without being able to prove a direct link to enrolling more students or raising more money for the foundation.
I do also recognize that each of us in the marketing field is trying to be more creative than our competition in getting our message out to our audience. This latest advertising space is certainly creative; I am just not sure how effective it is. Thoughts?
If you are really bored, you can watch the video of the Washington State University plane being painted with the school’s fight song in the background. 
I Have Seen It All…Facebook Credits for Sale at Target!
You could have picked my jaw up off the floor in the checkout line at Target. I don’t know why I allow myself to be surprised by things like this anymore, but I am. Right next to the gift cards you could purchase from Applebees, Target, iTunes, etc., was a gift card to purchase Facebook credits. Yep, you could give as a gift more reasons to waste time on the most popular social networking site. Amazing isn’t it that folks now actually spend money on Facebook for their additions to Farmville and other games/applications I routinely block.
So if you were still skeptical that people were spending inordinate amounts of time on Facebook being lured into the games and third-party applications out there, are you convinced now? Are you going to take advantage of their ad-serving platform (remarkably similar to the Google Adwords platform) or perhaps build a third-party application for folks to get addicted to?
What I’m Reading
I like to see the Standing Partnership “What We are Reading” list, so I thought I’d borrow that idea, and put my higher education lens on it!
- We are in the process of hiring graduate assistants in our unit, so I was on the hunt for writing tests! PRSA had a few tips on how to select the best writer for a job.
- Obviously, we are dealing with the millenials, Gen Y, whatever you wish to call them today. Many of us are hiring them at our workplaces. The Wall Street Journal published a very interesting article examining what a disservice social networking is doing for them as they enter the corporate world: Why Gen-Y Johnny Can’t Read Nonverbal Cues.
- Is blogging really good for your business? Hubspot quantifies an aspect of the ROI of blogging for you! Hint: small businesses that blog have 55% more website visitors!
The Rationale for Centralized Marketing Structure in Universities
I recently drafted a memo internally discussing proposed organizational changes. Though targeted toward higher education, the points are well made for any industry:
The rules have changed for marketing and communications professionals both in general, but especially at the university level. We are challenged to keep up with constantly evolving technology and consumer behavior patterns, even more so with the advent of the Internet and the explosion of social media marketing tools. The good news is that we have greater and more direct access to our customers, reducing the reliance on the traditional news media to communicate with our audiences.
The field of traditional public relations has evolved such that greater skills than just crafting the attention-grabbing press release are critical to job success. Public relations materials are now for more than just mainstream media audiences. Our audiences do not just want to see our organization on TV or in the newspaper; they want to see us on the web. Marketers are now challenged with helping customers move through the decision-making process with great online content; providing authenticity, not spin; and encouraging participation, not generating propaganda.
Communicating in the 21st century requires us to consider the following factors:
- We are in a competitive, global marketplace
- This is a 24/7 media environment—not just the traditional 9a-5p
- The marketplace is extremely consumer-centric
- Consumers have an expectation of governance and fiscal responsibility
Challenges we must overcome include providing context and perspective, demonstrating our competitive advantage, and ensuring our customers have a consistent message and experience.
In the traditional PR environment, organizations or units at a university could effectively function with a single individual generating press releases and responding in a reactive media environment. Today, teams of professionals are better suited to proactively tell the stories of success, and distribute those stories via all appropriate channels: through the website, video, publications, social media outlets, traditional media outlets, direct-to-consumer communication, and so much more.
All of these advances in the marketing and communications field present opportunities, but also the challenge of remaining professionally current in the field. Being part of a larger team or unit allows an individual to participate in more non-traditional professional development by learning from their teammates.
Given the fiscal challenges of both the federal and state governments (especially my home state of Illinois) and the tough budget times we face ahead, we must find ways to be more efficient and take advantage of existing resources—now more than ever before. We no longer have the luxury of separate marketing teams for various areas of the University. In addition to meeting the new challenges in the marketing field, it is fiscally more responsible to have one consolidated group that can address the marketing needs of various areas and the University as a whole.
Engaging Presentations
Last night I attended the Meet the Design Teams event for The City + The River + The Arch Competition at the Roberts Orpheum Theater in St. Louis. They have narrowed the field down to five design teams who are vying for the chance to design the landscape around the St. Louis Arch grounds on both sides of the Mississippi River and do a better job of incorporating this iconic piece of public sculpture into the city of St. Louis and it’s tourism industry.
I have to say it was very refreshing to watch Joe Buck as the emcee. He really injected some humor into it. He came right out of the gate saying that this was going to be a fun event rather than a formal event and almost took more of a approach of a traditional “roast” event rather than a formal presentation. So, immediately I was heartened to see that it wasn’t going to be a long boring night of presentations.
However, when the first design team got up to present their capabilities and their team members, I was a little disappointed the he was not as engaging a speaker as Joe Buck. I wasn’t expecting him to be a perfect speaker. However, I was asking for it to be a little bit more engaging and interesting and to capture my attention.
I think a lot of it had to do with the fact that the presentation set-up in the theater was poor. The image quality on the projection screen was dark and blurry at best. I don’t have the best eye sight but I don’t need glasses, and sitting in the back of the theater it was extremely difficult if not impossible to read any of the text on the slides and to make out the specific details and some of the images. And, his presentation was heavy on photography and imagery. So, I found that to be distracting and I lost interest very quickly in what he was saying because, I struggled to see what he was referring to on the presentation slides.
So, I guess my take away from this event was the importance of being an engaging and dynamic speaker. To capture the attention of an audience that can range anywhere from people who are specifically interested in architecture and design to just the general public interested in what’s happening in the community. You have to find that perfect balance between getting very technical to please the technical people but, also being very engaging and top of a strategy oriented to draw in the general public.
I also understood the importance of really understanding the venue in which you are presenting and the tools that are available for you, and tailoring your presentation to meet those challenges. I think that if any speaker would have paid close attention to the venue and the equipment available, they probably would have tailored their presentation a bit differently to meet those equipment challenges. And therefore, would have done a better job of engaging people.
Some of the best presentations I’ve seen interacted with the audience and really drew people with what interested them into the presentation. I know that it’s difficult to do with a large crowd but I think it’s possible.
So, my challenge to myself and to all of you is: What are you going to do to make each presentation different and unique and draw in that audience?
Your Customers Telling Your Story
I am sitting here in Clayton Studios working with Dick Ulett and Brenda Bertts Long to record the radio ads for SIUE. Our first campaign about a year and a half ago told the stories of success at SIUE through the eyes of faculty and students. We had such a quick turnaround time that we chose to use professional voice talent.
This year, we decided to tell the stories of success literally through our students and faculty. We asked for nominations from our staff and faculty of students who had great stories of success to tell. We then auditioned everyone, and selected a great pool of students to be the “voices of SIUE”.
I have to admit, I was extremely nervous about the level of talent we would find in the student body. Boy was I pleasantly surprised! We have amazing talent on campus for radio voices! Now we are in the studio today recording the ads with our students. It is amazing. They are so inspiring and have such great stories to tell with such energy and excitement! Talk about authentic—most places strive to bring this out of their actors. We are getting it right out of the gate!
Having your customers tell your story for you (and yes, students are our customers!!) is so powerful. I am excited to also see how this will play out in the rest of our media—billboards, print ads, blogging, and so much more. I really do hope that this strategy will resonate with prospective students and parents—hearing the stories from our students.
And by the way, Brenda Bertts Long, Dick Ulett and Clayton Studios are absolutely amazing and great to work with! I highly recommend them.
Spray and pray
I read a very interesting blog post updating me on the new terminology describing unmeasured direct mail as a part of your overall marketing and advertising strategy. It’s now called “Spray and Pray,” and defined by The Agitator in the context of fundraising:
“…the thoughtless, non-strategic practice of tossing as many appeals and acquisition pieces as an organization can afford — regardless of long-term result…”
Let’s put this in the context of higher education student recruitment and fundraising. The phrase is obviously referring to the way direct mail is handled: you spray pieces, cast a wide net to a not necessarily highly qualified audience, and you pray they take whatever action you’re asking of them in the direct mail, whether that be to contribute money to your alma mater, attend an event, request more info, or even enroll in classes. The problem here is that you can only pray they do what you asked.
Another option is implementing tactics such as Google AdWords or other search engine marketing tactics; obviously Yahoo, MSN, and others have their own ad serving platforms. Facebook even has an ad serving platform that is very similar to Google’s.
In those instances, you define who your target audience is, what is a qualified lead or prospect to you, and make sure that your ad is served only to those prospects. In addition, you’re serving ad copy that matches what they have searched for. For example, if someone is searching for St. Louis MBA programs, you’re not going to serve them ad copy that addresses the School of Education program’s offerings. You’re going to give them ad copy tailored to what they’re looking for, which is an MBA program. You’re also going to take them to a landing page on your website that also addresses specifically what they are looking for.
Now, I understand that there have been improvements in direct mail technology, such as print on demand, variable data fields and personalized URLs for tracking responses where you can really try to personalize a direct mail piece based on what you know of your customer and their preferences, but direct mail still is not as interactive, measurable and successful as a search engine marketing campaign.
So why then do colleges and universities still spend hundreds of thousands of dollars to purchase names and ‘spray and pray’ with direct mail campaigns to prospective students and donors without even investigating the success of search engine marketing campaigns, such as Google AdWords? Even if you just do a test run, side-by-side to determine which one has a better return on your investment. I have asked countless groups of people I have presented to a series of four questions:
- When is the last time you went to the printed yellow pages, looked up a product or service and made a purchase?
- When is the last time you made a purchase based on a radio, TV or print ad you have seen?
- When is the last time you made a purchase based on a piece of unsolicited direct mail you receive?
- When is the last time you opened up your browser, went to Google or Yahoo, searched for a product or service, and ultimately made a purchase on line?
You can guess what the funnel of number of hands raised through each question. How many times do I have to ask the question before someone listens and sees value in SEM? Why is higher education different than any other information gathering sales process?
Most productive CEOs
I recently read an article in Inc. magazine that featured several CEOs talking about work habits that have contributed to their success, or helped them be more efficient, in their particular business environment. The first CEO that caught my attention was Caterina Fake of Hunch who talked about keeping her schedule completely free-form, and working on what she wants to work on, whenever she wants to work on it—whatever seems right at the time. That approach would be great if you only had to work on certain things and could delegate everything else to a person below you. But for me, her approach doesn’t provide a good suggestion of how to help prioritize my day, or do a better job of delegating, so that I can sit and focus or work on the things that I want to work on at that particular time. After testing that strategy for a few days, I would up with a lot of work that didn’t get done during the week, waiting for me to do over the weekend. That isn’t helping me achieve work-life balance. Perhaps I am not delegating well enough.
One interesting, great thing she mentioned in the article is that when she conducts her staff meetings, everyone is standing during the entire meeting. Also, everyone drinks 16 oz. of water before the meeting starts, and the meeting is over when the first person has to go to the restroom. To me, that seems like an incentive to keep your meetings short, to the point, brief, and not waste your time meeting for the sake of meeting, but rather get out there and get something done for your company. Go build a relationship. Go check an item off your to-do list. I brought that suggestion to the next regular staff meeting I have to attend. Everyone seemed to agree–let’s see if it gets implemented.
The other CEO that caught my attention in this article was Mark Cuban of the Dallas Mavericks, who claims that he only conducts business and correspondence through email. He does not utilize voicemail or meetings. If you do require his attention in a meeting or by voicemail, it better be something very good and important…likely conducting millions of dollars of business.
Now, there’s something lost for me when you communicate only exclusively through email. First of all, you assume, and insist, that everyone you deal with also operates through email. To me, a sign of a good manager of people is that you adapt your communication style to fit the needs of those working for you. My job is to bring out the best in people and help them achieve their potential. So if, for example, someone’s personality or style is not conducive to business via email exclusively, are you isolating them? Are you potentially surrounding yourself with people who are like you, rather than surrounding yourself with a rich, diverse mix of people who bring different trades, ideas, thoughts, and processes to the table?
I think an emphasis on diversity brings a richer mix of ideas and innovations to the table. But then again, Mark Cuban has made quite a bit of money and is quite successful at what he does. I guess, once you get to that point, you can tell everyone around you how you prefer to do business. But in the meantime, those of us who are still stuck in the middle and trying to work our way up will need to continue to adapt to the communication styles of those who are reporting to us. Also, you can’t tell inflection, personality, or emotion through written, email correspondence. There’s something to be said for working either via the phone or in person, being able to get to know someone just a little bit better.
I also ask, how do these ‘unique’ work styles affect relationships with the customer? If you will only communicate with them via email, or you won’t commit to a schedule or timeline, is that helping further the relationship or to begin a new relationship? Unless you are in a business where your customers have no choice but to do business with you, I would suggest that you be willing to meet your customers where they are comfortable conducting business and communnicating with you. That will likely lead to a multi-layered communication strategy to address the varying needs.
About Me
I am currently the president and general manager at Access Marketing Company. You can read my full resume on my LinkedIn profile. Some of my favorite accomplishments include canning and preserving much of my summer vegetable and fruit garden, being part of the St. Louis Business Journal Class of 2007 Top 40 under 40, and continuing to play beach doubles volleyball – even placing in an EVP doubles tournament.
Subscribe
Beth Keserauskis 