Every year there are lists developed to rate which is the best airline to fly with. Some are prestigious lists that may come with awards and valuable media attention. Given the importance of these lists it is interesting to look at the different ways that they assign ratings to airlines. This is especially important since the objectivity of some ratings have been questioned. There have been suspicions from within and outside of the aviation industry that some of these lists could be improperly influenced by airlines who want a shortcut to improved ratings.

One site that is fairly transparent about their method is usebounce.com (link to the airline rating page below), which is a company that offers luggage storage services. Their ratings focuses on airlines in the US, domestic and international ones flying into the US. Their rating system makes use of on time arrival (for domestic airlines), complaints, service, meals, seat comfort, inflight entertainment and baggage allowance. They also specify from where they get the data for these parameters, which is basically from government and other open sources.

For domestic US airlines it is Delta, Hawaiian and Horizon who take three top positions out of the twelve domestic operators on the list. At the bottom are Spirit, Allegiant and American. Even with the transparency on the page of how the rating is done, there are however questions to be asked, such as how the nature of the parameters are considered and how the different parameters are weighted against each other. As an example, counting simply the number of complaints would disadvantage airlines which transport more passengers, unless the complaints are expressed as a ratio per sector or kilometers flown. Arrivals on time was taken from July of last year, which may be a month in which a certain segment of passengers travel more, which may be the disadvantage to airlines that cater to that same segment. The service, meals, seat and inflight entertainment are based on reviews submitted too the website airlinequality.com, which does not say much about methodology. It is however based on passenger reviews but a lot of questions remains how balanced and robust this system is in regards to comparing different amounts of complaints and being able to withstand campaigns of manipulation.

The three top positions for best international airline were taken by All Nippon Airways (ANA), Singapore Airlines and Korean Air. It seems fair to wonder if the fact that three Asian airlines took the three too spots may be linked to cultural aspects, such as that passengers from these countries may be less likely to complain than passengers from other countries. This leads to further thoughts about how reliable these kind of ratings are mmost of them seem to have quite a naive trust in numbers. It should be noted that even the well established ratings of World Airline Awards – SKYTRAX (link) have been questioned (link), with some airlines withdrawing from the rating system (link). Simply the fact that Skytrax offers consulting services to airlines and airports should be enough to be suspicious of its ratings, especially as there have been examples hurting the credibility of them. One such example was when Hainan Airlines in 2011 received a 5-star rating concurrent with a consulting project.

In summary, what this little example shows is that even when there is transparency about a methodology that looks sound at first glance, there are many more questions to ask about any rating system for airlines. There seem to be a lack of integrity and scientific rigour missing for these systems, at least if they should be given the trust of potential passengers. These ratings systems will continue to attract media attention but those of us who work in and may be affected by them should be asking for something better than the rating systems which are around today.

Link to article:
The 2021 Airline Index