CMS Critic

My Take on PacktPub’s Open Source Awards

By on · 12 Comments

This year I chose to sponsor PacktPub’s Open Source Awards by publishing and sharing the news of the various steps in the voting and nomination process. In return for the CMS Critic logo being posted on the awards page, I would publish their news releases as the process progressed.

Unfortunately, I have grown to regret offering to act as a sponsor. One of the things that I’ve noticed this year more so than others is how poorly organized these awards are. They seem to serve more as a marketing tool for PacktPub to drive traffic to their site than to add value to the community. Sure, there are monetary values attached and it’s great that these prizes are awarded to worthy projects but there are also serious flaws in how these awards are run.

Take for example the “Best CMS for Ease of Use”  category.

One would expect a CMS that is actually easy to use and has a history of being easy to use to win this, right? Guess who won this year? Drupal.

Drupal is a great project, but if there is one thing it is not known for, it is ease of use. Drupal has to be the absolute last CMS that I would ever name in this category. It’s not shooting to be the easiest, but rather the most versatile and that is one of it’s strengths, but to give it this award is just plain ridiculous.

I am not attacking Drupal. I am a huge fan, but even the members of the Drupal community have to be honest and admit that this is a bit far fetched. I chose to use it as an example because it’s a very good one.

There are tons of open source projects out there that do not have huge communities of people that can vote for them and this is one of the ways these awards fail. They rely on voting from the community.

I’m all for giving the public a chance to vote but let’s be realistic here. If you pit the huge community of Drupal, Joomla or WordPress users against another smaller project, who do you think will win in sheer volume of votes? Does this mean it’s indeed a good indication of which CMS is best? Not at all.

Advertisement

This isn’t just about CMS though. This is about the awards as a whole. Most of the winners are well known projects that have been well established for quite some time: JQuery, Blender, Gimp. All wonderful projects with huge communities. There are a few that won that are lesser known and I’m very glad they got the recognition they deserve such as ImpressPages, for example, but they are the exception to the rule.

I don’t want to sound like I’m putting down the winners of these prizes. I think it’s great that they are getting cash for their projects and there are plenty of great projects listed. I simply do not think the awards themselves add any real value other than to contribute money and provide Packt with some extra traffic for the 3 or so months they run.

As I mentioned, CMS Critic acted as a sponsor this year and, as expected, Packt kept in touch with me to let me know of the various stages and how they were progressing so I could publish stories keeping the community informed. Then came the time where the winners were announced and I received not a single email to inform me of any winners, no press releases, nothing. In fact, I completely forgot about the awards until I saw a post somewhere else mentioning the winners. Yes, it’s partly my responsibility to keep on top of these things but one would also expect that they would be in contact to ensure the proper amount of promotion is given to the winners, and I didn’t hear a word.

Perhaps I’m expecting too much, I don’t know. What I do know is that I won’t be sponsoring next year.

I’m not here to just rant (although I do it well). I’m here to offer suggestions on how I would fix it as well, and here they are:

  1. Implement proper judging by a group of known open source advocates and experts in their respective industries.
  2. Maintain voting for a “People’s Choice” set of awards for purely recognition purposes with no monetary prize assigned.
  3. Bring in some other sponsors to up the prizes amounts and allow for more competition.

What do you think? Do you agree? Disagree? Do you have a better idea? I’d love to hear from you below.

 

Tagged with → • • • • • •  
Author

Mike Johnston

Mike is a CMS Analyst & Consultant who founded CMS Critic in 2008. He regularly consults with companies to assist them with the CMS selection process and is available for speaking engagements. You can contact him here or follow him on Google Plus.

Visit Authors Website →
Authors Google Profile →
Share →
  • http://www.mojoportal.com Joe Audette

    I never used the word “merely” but I maintain that it certainly is a marketing campaign and Packt gets marketing value out of this. Are you denying that it is a marketing campaign for Packt?

    I think Packt does have their heart in the right place but it is not all one sided gift giving. Just look at how it works.

    First phase is nominations and opensource CMS vendors need to get enough nominations so they put up banner ads on their sites, in their blog posts, in their newsletters, linking to the Packt site to get as many nominations as possible.

    Next phase if you got enough nominations you’re a finalist and now its the voting phase, so now we use another banner ad to drive traffic to the Packt site to get as many votes as possible. In years past Packt even helped create the banners for the finalists.

    Finally the contest is decided by judges and most years very little interaction between the contestants and the judges and very little explanation about what criteria or evaluation process was used to decide.

    But if you win, then of course you make another victory ad on your site linking to Packt for the whole next year. We did this after winning Best Other Open Source CMS in 2007. It gave us bragging rights and it continued to drive traffic to the Packt site. It is only natural if you win to make as much hay out of it as possible, but only if you win does it really turn out to have much marketing value for the open source project that participated.

    So the mecahnics of the contest all result in lots of advertising for Packt, the only difference is that the pay for the ads only goes to the winners in the form of the reward money. Certainly it would make sense that the reward money comes out of the Packt Marketing budget just like paid ads do.

    From my point of view it would be just as much support for open source projects to buy advertsing on the web sites for open source projects.

  • http://martinaspeli.net Martin Aspeli

    Having worked with Packt as an author, my impression is that their hearts are in the right place, they’re just incredibly disorganised at times.

    I love their sponsorship of open source and the fact that they’ll publish books on topics that many other publishers consider too niche. They also very actively recruit authors. However, they are frequently their own worst enemy, e.g. by not realising that someone has already written a book for them when being contacted for another purpose.

    I think the awards are similar. Yes, they’re a marketing tool. Why else would Packt do this? I think there’s lots of room for improvement, too, but in the end, all awards have some kind of bias and must be taken with a large grain of salt.

    Martin

  • http://www.PacktPub.com Julian Copes

    Hi All,

    My name is Julian Copes and for those of you who do not know, I help to organize the Packt Open Source Awards along with members of my team.

    As always, Packt is open to feedback on the Open Source Awards. This has been demonstrated in the past when we changed the Awards from the CMS Award to the Open Source Awards following a vast amount feedback that projects (other than CMSes) were keen to get involved in the Awards..

    We take on board criticism and the comments on the Awards this year will be taken in consideration when we decide what we want to do with the Awards in the future.

    As always we will look at how to improve the Awards and introducing “Most Voted” awards alongside(and/or separate to) “Best quality product” awards (@ sigurd) is an interesting idea which we will definitely discuss.

    Regarding the idea the Awards is merely a marketing tactic for Packt, I think that’s a bit unfair. Packt has paid over $100,000 to Open Source projects that have won Awards and both Drupal as well as WordPress have named the Packt Open Source Awards as a significant yardstick in its project growth.

    We’ve always stated the Awards ‘is a contest that aims to encourage, support, recognize and reward Open Source projects’ and have never claimed to be the user’s definitive guide to choosing software.

    A part of our goal with the Awards is to encourage new projects and to counter the difficulty of new smaller projects competing with more established ones and this is why we introduced the Most Promising Open Source Project category.

    If the Awards is a marketing tactic, it is a marketing activity for Open Source projects, aiming to bring exposure to growing and established Open Source projects.

    Packt’s intention here for the Awards is to recognize and encourage Open Source projects and I see it as a lighthearted, fun, celebration of progress and in some cases, comebacks. In addition to that, we then help Open Source projects that could do with financial support by providing a prize fund for all key Award categories.

    While I am all for constructive criticism of the Awards and ideas of how to ensure that the Awards reflect the growth of Open Source software and the complexity in its market (e.g. one size doesn’t fit all), I do rebuff the idea that the Awards is merely a campaign to sell products or get Packt exposure.

    Also- I will take the comments towards the organization of the Awards on board in order to improve the handling of the judging process, and the communication with judges as well as project representatives.

    I would like to thank you all for your feedback and express my genuine appreciation for the comments. I am a huge advocate of continuous improvement (Kaizen) and I hope we’re able to improve the Awards in the future.

    Julian

  • http://www.idatix.com Samantha

    I have to agree with you as well, Mike. It takes courage to stand up for what you believe, even if it is ‘only’ a CMS award show. You have to represent and support the things you will be proud to be associated with, so I commend you on your honesty. Nice job.

  • http://librecad.org Ries

    Mike,

    I will agree with you. This type of voting is nothing more then a popularity contest (who has the most users), in the same line as American Idols other other voting type of contests (don’t ask me how I know that name please!!!).

    What I don’t really understand is that Packt does have a large address book of book writers, and they could at least these people to take a look at 2 or 3 CMS systems and make an objective review about the CMS systems they asked to review.

    I have never voted on this things though, but I do want to say I full agree with your blog post, and packt need to change there voting methods.

  • http://www.lab2net.net samy

    Hi
    Since those awards are based on voting … they doesn’t mean that one project is the best … but the most popular if a big part of the community vote.
    And were is WORDPRESS?

  • John Coonen

    Mike, Takes nads to say what you said. That’s why I like ya.

  • http://www.silverstripe.org Sigurd Magnusson

    Mike, well done to speak your voice on this, especially after having sponsored it. My take is that the issue is with that the naming and process of the awards are at odds with each other.

    I’d have no issue if they produced two streams of awards. I.e. ask the open source community to create a set of “Most Voted” awards and separately asked some qualified judges to create a set of “Best quality product” awards.

  • http://www.deanmarshall.co.uk/ Dean Marshall

    I’ve always resisted the encouragement to vote in these awards/contests. As an end user (and third party support consultant) of one of the larger CMSs I’ve always felt that the push to vote for the project at best turned such awards into a popularity contest and at worst rendered them totally without value and bordering on corrupt.

    It is useful – and valuable – to have some promotion within the industry – but the key goal of awards should be the promotion of (Free) Open Source Software not simply the promotion of organisations running the awards.

    I don’t intend to be anti Packt – and have enjoyed the spectacle over th last 7? years I’ve been aware of the awards – but I will jump on the bandwagon calling for restructuring of the awards and the way the winners are selected.

  • http://www.mojoportal.com Joe Audette

    As far as I know those sub categories like Ease of Use, Performance, Best .NET CMS etc were not even there during the voting phase. I think they just re-used some text from a previous year contest on their winner announcement page. For example DotNetNuke was not even a finalist this year and they are listed as best .NET CMS which they won a few years ago but that was not even a category this year and as far as I know they did not participate this year.

    The whole thing was very sloppy this year, they didn’t announce the winners on time according to their own dates and seemed to just quietly put up the results over the weekend.

    mojoPortal was a finalist again this year and has participated every prior year since 2007 but it has always been obvious to me that it is mainly a marketing campaign for Packt. Its the kind of thing where if you win you can use it as marketing fodder and pretend its the most important contest in the world and make a big splash about it, but if you lose its really no big deal. Last year was the only year we ever received any questions at all from the judges, no interaction with them at all this year or other years. Nothing against Packt, it is a clever idea for a marketing campaign and it was fun for a while but we don’t intend to participate if they have it again next year. I think Bryan Ruby was right last year when he questioned the value of this kind of contest and I agree with him that those that didn’t win shouldn’t be considered losers. http://cmsreport.com/content/2011/01/judging-five-open-source-content-management-systems

  • http://thenewhut.com Steve Jarvis

    I couldn’t agree more, the Packt awards are nothing more than a publicity contest. I fully agree a set of experts should be brought in to vote.

    • http://www.cmscritic.com Mike Johnston

      Thanks for the comment Steve, I was curious if others felt the same or not..