Elizabeth Keserauskis

Building relationships and making connections

Forbes College Ranking: True Gauge or Sketchy Data?

Forbes recently released their annual rankings of colleges. The rankings were calculated in partnership with the Center for College Affordability and Productivity. My institution was conspicuously absent, and I was asked to investigate why. (The responsibility for responding to external credibility surveys lies within my unit.) What I have subsequently found is that the rankings are based on existing, publicly available data (IE no surveys were sent to each institution requesting data, information.) Which in and of itself would not trouble me, except that the data points used, to me and others, are questionable at best. Additionally, information like the infamous “other cost” category institutions report as an allowable cost at the discretion of the student to cover expenses like mileage to clinicals, internships, etc. in their financial aid package, are treated as actual billed charges, therefore increasing the “cost of attendance” and subsequently this elusive “net price” calculation and the debt load calculation (predicted, I might add).

I have several excerpts from articles, blog posts, and even their own methodology posted below giving a glimpse of the, in my humble opinion, “sketchiness” of this ranking. I can only hope that Joe and Jane Sixpack are able to sort through the variables…oh wait, they likely can’t. So now we have another “think tank” with a clear agenda (political or otherwise) leveraging a brand like Forbes to advance their cause.

How important are rankings like this and U.S. News and World Report Best Colleges? Only the audiences we are trying to attract will tell. And believe me, I intend to ask them just that, so we can tailor our approach accordingly to these surveys.

Compiling the Forbes /CCAP Rankings (excerpt from the methodology document, full document can be found on their site)

By the Staff of the Center for College Affordability and Productivity

 Ranking Factors and Weights

The Center for College Affordability and Productivity (CCAP), in conjunction with Forbes , compiled its college rankings using five general categories, with several components within each general category. The weightings are listed in parentheses:

1. Student Satisfaction (27.5%)

  • Student Evaluations from RateMyProfessor.com (17.5%)
  • Actual Freshman-to-Sophomore Retention Rates (5%)
  • Predicted vs. Actual Freshman-to-Sophomore Retention Rates (5%)

2. Post-Graduate Success (30%)

  • Listings of Alumni in Who’s Who in America (10%)
  • Salary of Alumni from Payscale.com (15%)
  • Alumni in Forbes/CCAP Corporate Officers List (5%)

3. Student Debt (17.5%)

  • Average Federal Student Loan Debt Load (10%)
  • Student Loan Default Rates (5%)
  • Predicted vs. Actual Percent of Students Taking Federal Loans (2.5%)

4. Four-year Graduation Rate (17.5%)

  • Actual Four-year Graduation Rate (8.75%)
  • Predicted vs. Actual Four-year Graduation Rate(8.75%)

5. Competitive Awards (7.5%)

  • Student Nationally Competitive Awards (7.5%)

School Selection

The 650 institutions of higher education in this ranking are schools which award undergraduate degrees or certificates requiring ―4 or more years‖ of study, according to the U.S. Department of Education, and only those schools categorized by The Carnegie Foundation as Doctorate-granting Universities, Master‘s Colleges and Universities, or Baccalaureate Colleges are included in this sample of schools.

Of the 650 schools included in the sample, 608 wereincluded in the 2010 college ranking. (A total of 610 schools were ranked in 2010, but two of them, Bryant University and Missouri University of Science and Technology are now classified as ―Special Focus‖ institutions by the Carnegie Foundation). We have accounted for any name changes that have occurred over the past year. The 42 schools added this year to the sample are all institutions classified by the Carnegie Foundation as Doctoral/Research Universities and were added based upon undergraduate enrollment size.

A Little History of the Forbes Rankings from 2008-present, excerpt from a commentary on methodology (full commentary can be found at: http://bestcollegerankings.org/popular-rankings/forbes-college-rankings/)

2008 marked the first year that Forbes entered the college ranking fray. They choose to use a methodology that included the following percentages: Listing of Alumni in the 2008 Who’s Who in America (25 percent); student evaluations of professors from Ratemyprofessors.com (25 percent); four-year graduation rates (16 2/3 percent); enrollment-adjusted numbers of students and faculty receiving nationally competitive awards (16 2/3 percent); average four year accumulated student debt of those borrowing money (16 2/3 percent). They did not break colleges down into different schools as U.S. News does, but instead choose to separate private and public colleges instead.

Methodology: In conjunction with Dr. Richard Vedder, an economist at Ohio University, and the Center for College Affordability and Productivity (CCAP), Forbes inaugurated its first ranking of America’s Best Colleges in 2008. They based 25 percent of their rankings on seven million student evaluations of courses and instructors, as recorded on the Web site RateMyProfessors.com. Another 25 percent depended upon how many of the school’s alumni, adjusted for enrollment, are listed among the notable people in Who’s Who in America. The other half of the ranking was based equally on three factors: the average amount of student debt at graduation held by those who borrowed; the percentage of students graduating in four years; and the number of students or faculty, adjusted for enrollment, who have won nationally competitive awards like Rhodes Scholarships or Nobel Prizes. CCAP ranked only the top 15 percent or so of all undergraduate institutions.

Negative Commentary on the Methodology (Excerpt from Suite101.com: The Forbes Best College Rankings 2011: Are They Kidding?

What Goes in Must Come Out

First of all, a quick review of the Forbes methodology. It is the goal of the rankings to evaluate college as a consumer or investor would evaluate a commercial product. The focus is on the return on investment–for what you pay, do you get a good “value”? The most important element in assessing this value is “Post-Graduate Success,” accounting for 30 percent of the total.

This “success” is measured by the salaries of graduates as reported by Payscale.com; membership in “Who’s Who”; and by alumni representation on a list of corporate officers chosen by Forbes and the Center for College Affordability and Productivity (CCAP). CEOs and board members of leading companies are the only persons who are eligible, thereby narrowing the definition of “success” to achievement in the business world only.

It is interesting that Forbes would allow use of “Who’s Who” listings as a measure of college success. In a 1999 article for the magazine called “The Hall of Lame,” Tucker Carlson, a Fox News commentator, derisively showed how inclusion in Who’s Who publications did not require notable achievement.

Another 17.5 percent of the total is based on student evaluations of instructors, taken from the website Ratemyprofessors.com. While student evaluations are useful, they can also lead professors to emphasize popularity at the expense of scholastic rigor.

An additional 17.5 percent of the total comes from actual and anticipated four-year graduation rates. Using four-year rates rather than six-year rates clearly favors colleges that are wealthy enough to subsidize virtually all eligible students based on need or merit, or whose student body is made up of highly-prepared students with sufficient economic support. State universities, whose students often have to work part-time or even take a semester off from school, usually cannot match the four-year graduation rates of private colleges.

Likewise, the rankings penalize colleges whose students have higher student debt loads, and this also slants the rankings toward wealthy colleges and parents.

Academic Reputation—Forget It

The most glaring deficiency of the Forbes survey is that the only standard it uses to assess the intellectual credibility of a college is the data from Ratemyprofessor.com. Academic reputation and faculty achievement count for nothing, even though a recent UCLA study of more than 200,000 freshman across the country revealed that undergraduate academic reputation was the most important factor for these students when they were choosing a college. Forbes wants to change that perception, but does the magazine really believe that reputation counts for nothing in the business world as well?

It is ironic that a survey that is supposed to be student-centered disregards the one factor that students themselves cite as being most important to them: quality. Interestingly, the UCLA study also showed that prospective students are learning to be guarded in their use of college rankings, a healthy sign indeed.

August 10, 2011 Posted by | higher education, marketing, reputation management | , , , , , , | Leave a comment

Agility

After a particularly frustrating series of days working for a state institution, I was reflecting on the concept of agility and how important it is for an organization. More appropriately, I was ranting about how NOT agile a public higher education organization is. So I decided to Google “business agility”. And since Wikipedia is clearly the destination for accurate information (*sarcasm*) and you really should believe everything you read on the Internet (*more sarcasm*), it was only natural to include information from their entry (which, by the way, is cited as Dyer, L. and Ericksen, J. (2009). Complexity-based Agile Enterprises: Putting Self-Organizing Emergence to Work. In A. Wilkinson et al (eds.). The Sage Handbook of Human Resource Management. London: Sage: 436–457.)

Agility is a concept that incorporates the ideas of flexibility, balance, adaptability, and coordination under one umbrella. In a business context, agility typically refers to the ability of an organization to rapidly adapt to market and environmental changes in productive and cost-effective ways.

Where to begin pointing out the ways we are not agile? I often feel like each day I am trying to change the course of a cruise ship with a canoe paddle. All frustration with the business practices required by the State of Illinois aside, let’s focus on the communication side of a business.

We cannot craft messages and responses by committee anymore. The world is moving too fast and the situation will have changed and incorporated more information faster than the statement can be reviewed by 3 of the required 7 people. What about establishing the core values, mission, brand platform and promise, and overarching key messages, and then deputize people to be company ambassadors at all levels?

Also, if your entire communication/sales strategy is based largely on printed material and direct mail, how up-to-date do you think the information is that people will have in their hands? I am not advocating banishing all printed material, as they have a time and place. However I am suggesting that our customers are so much more savvy than ever before and will go to the places where they want to consume information (websites, blogs, social networks, friend networks, etc) rather than try to figure out how a business is trying to force them to consume the information. Meeting the customers where they are looking for information requires agility in a business, and specifically in the marketing and sales areas. It also requires the leaders of both areas to buy into the concept, versus continuing with “the way it has always been done”.

Going back to the Wikipedia entry,

There are several key distinctions between the agile enterprise and the traditional bureaucratic organization. The most notable is the agile enterprise’s use of fluid role definitions that allow for dynamic decision making structures. Unlike the rigid hierarchies characterizing traditional bureaucracies, organizational structures within agile enterprises are more likely to fluidly adapt to changing business conditions into structures that support the current direction and any emergent competitive advantage. (Dyer, L. & Ericksen, J. (2007). Dynamic Organizations: Achieving Marketplace Agility Through Workforce Scalability. In J. Storey (ed.). Human Resource Management: A Critical Text (3rd edition). London: Thomson Learning: 263–280.)

How can we change the state and structure of public higher education (Illinois specifically for my knowledge base) to become more agile? To me that is the new million dollar idea. Wish me luck finding it.

June 25, 2011 Posted by | communication, higher education, leadership | , , | 2 Comments

New Implications for Your Job Search: 7 Years on File

All Facebook: the unofficial Facebook resource, posted an article recently (ALERT: Job Screening Agency Archiving All Facebook) about Social Intelligence Corp, which offers a background checking/screening service for employers. From their article:

The FTC decided Social Intelligence complies with the Fair Credit Reporting Act, the same set of rules that keeps your bill-payment records on file with the consumer bureaus for seven years, according to Forbes.


That means your antics documented, photographed, posted and tagged on Facebook will be available to your prospective and current employers for 7 years. I have heard comments from folks saying that isn’t it about time we stopped trying to pretend like people we really aren’t in order to get a job? Sure that would be the perfect world. But do you really want to be the trailblazer to set that trend? If so, your job options will be severely limited–best of luck with that.

So if you haven’t already appropriately adjusted your privacy settings and stopped posting the ridiculous and embarrassing photos and status updates, now is the perfect time. Get your head out of the sand! Seems to me given this development and the recent facial recognition technology addition, folks better start thinking more seriously about their online reputation (especially kids getting ready to enter the workforce in the next 7 years.)

June 22, 2011 Posted by | reputation management, social media | , , , , , | Leave a comment

Facial Recognition Technology = Creepy

I think I agree the title of the  PC World article: Why Facebook’s Facial Recognition Technology is Creepy. The more photos people tag of you or you load and tag yourself, the better the facial recognition technology gets at recognizing you in photos and making suggestions for folks to tag you. How is this not creepy?? It’s starting with Facebook, but will soon be everywhere. Those photos you thought were innocent of you doing keg stands in college? Even if you’ve untagged yourself, it is still findable online and attributed. And don’t say that you can “opt out” of the service…that just means people cannot automatically tag you. But that isn’t stopping the technology from gathering data and refining the ability to recognize your photo.

In addition to my own privacy, my biggest concern is kids/teenagers/college students who already think they are invincible online, and don’t think the first time (let alone twice) about posting compromising, silly photos of themselves on Facebook and other sites. I spend a lot of time with college students talking to them about how to manage their reputation online and to start thinking very early about what their image is online BEFORE they have to start looking for jobs. This will certainly be a part of my presentation from this point forward, but I am only hitting a small number of kids.

I love the suggestion she has in the article–opt out of the service immediately, and start randomly uploading pictures of inanimate objects and tagging them as yourself to keep Facebook guessing what you really look like!

June 15, 2011 Posted by | reputation management, social media | , , , | 1 Comment