Technology

Top Ecommerce Platform Surveys

Over at Creatuity, we just finished up our first eCommerce platform survey to determine the top eCommerce platforms in use on the web. First of all - huge thanks to Tom Robertshaw for starting the first eCommerce platform survey that I'm aware of a few years back - his most recent results are available on TomRobertshaw.net. I wanted to briefly talk about why we ran our own survey, why people run these surveys and how they work, especially after the recent survey that Aheadworks ran raised some questions on Twitter.

I decided to have the Creatuity team put together our own study/survey for a few reasons:

  1. Provide independent validation of Tom's survey.
  2. Implement improvements on the survey methodology being used by Tom and others.
  3. Because it provided an interesting challenge for us to tackle.

And, to be honest - everyone runs these studies because they make great material to share on social media and to encourage people to share and link back to your site. There's definitely a reason that everyone releases these studies in April - everyone wants to publish theirs before Magento Imagine, the annual eCommerce conference hosted by Magento, the platform that currently leads the web in usage.

As Tom mentions on his survey results, he co-owns an eCommerce development company that focuses on Magento, which could open him up to claims that he's biased. By developing our own study independent of Tom's (and in some cases providing differing results from Tom's study), I hope to provide validation that Tom's survey is accurate.

When reviewing the results from Tom's survey and the AheadWorks survey I saw a number of ways we could improve upon their methodology. Both of them survey the top 1 million sites on the Internet as measured by Alexa. The problem with Alexa is that they measure the top 1 million sites via data they collect from a toolbar they ask users to install. This toolbar only runs on Internet Explorer, Chrome and Firefox - ruling out users that use Safari or Opera. It's a limited data-set, and it has its own built-in biases due to the way the data is collected.

That's why our survey includes both the Alex top 1 million and the Quantcast top 1 million, Quantcast takes the opposite approach of Alexa - site owners install tracking code on their site, similar to Google Analytics, so the traffic data measures visits from all visitors, no matter what browser they're using, but only on sites that have the code installed.

I also noticed in both Tom's results from last year and the AheadWorks results, the most popular eCommerce solution for WordPress, woocommerce, wasn't included. This made me dig further into their results, and I realized there were a number of eCommerce platforms that weren't included. Tom has since updated his survey to include woocommerce, but Aheadworks still isn't including this platform, which makes up a significant portion of the eCommerce sites on the Internet.

So, we decided to put together our own study. Someone asked recently on Twitter if these surveys are based on downloads, sales, etc. - they're not - they're based on a technology similar to how Google builds its search index. We take the list of sites to study (the Alexa and Quantcast top 1 million sites, which, once de-duplicated, contain approximately 1.7 million sites) and load them into a database. We then connect to each of those sites, and scan it for a fingerprint. By a fingerprint, we mean a string of text that is unique to that specific platform - i.e., there are certain strings containing the word 'Varien' we use to detect various Magento versions. We maintain fingerprints for over 55 platforms at this time, and are constantly adding more. We then also check the site's IP address and the ownership data of that IP address to determine if the site is running on a cloud-based service (i.e., Shopify, Volusion, Magento Go, BigCommerce, etc.). These results are all loaded into a database, which we then analyze to generate our reports.

Of course, with 1.7 million sites if you assume it takes 10 seconds to connect to and evaluate each site, running this study on a single server would take over 6 months! That's where Amazon's AWS EC2 system comes in to play. We load our list of sites into an Amazon SQS queue and then run a master EC2 node that spins up 16-32 EC2 instances that split the queue up and process everything in under a week.

If anyone has any questions about our study, feel free to post them here or find me on Twitter at @JoshuaSWarren.

Magento URL Key Optimizer Userscript

Do you duplicate products in the Magento admin interface? If so, you've noticed that Magento doesn't change the 'URL Key' field by default, which means every time you duplicate a product, you have to edit this field manually. If duplicating products is the main way you add new products to your site (and for many Magento users, it is), you're spending way too much of your time making this manual change, when Magento should be doing it for you. I've put together a quick userscript (so this requires Greasemonkey for Firefox or Chrome -> see Userscripts.org for more information on how to use a userscript) that checks if you're editing a Magento product's title, and if you are, it automatically updates the URL key. This solves the case of needing to update the URL key manually while duplicating products, and has a nice added bonus that any time you change a product's name, the URL key will be updated automatically.

The current version does some super-basic optimization on the URL key as well to make it a bit more Google-friendly, and I plan on updating it as time allows to enable automatic advanced search engine optimization of the URL key.

Install the script from Userscripts.org, or submit a pull request on Github to contribute!

Is your iMac locking up, freezing or beach balling?

Some time ago, I added a Crucial m4 256GB SSD to my iMac. The performance boost was even better than I expected - the iMac now writes data about 2.5 times faster than the stock hard drive, and it reads data over 5 times faster - considering how much time I spend working in front of a computer, this really improved my day! What didn't improve my day, however, was that some time ago, I noticed that my iMac would never wake back up after it went to sleep; I might get as much as the login screen to appear, but the time would be wrong and the password prompt either wouldn't appear or wouldn't function. I feared the worst - that the SSD or the original hard drive were failing. I tried changing the power saving settings, disabled powering down the hard drives when they weren't in use, etc., but the problem kept occurring. Even if I prevented the computer from sleeping using the free utility Caffeine, it would still lock up at some point.

I was about to disassemble the iMac to replace the SSD (which is no simple task - it requires using suction cups to remove the screen, disconnecting the entire LCD and disassembling basically the entire computer) - that's when I decided to try Google'ing for the problem one last time. Yet again, I couldn't find anyone with the same symptoms as my iMac - specifically, beach balling on the wake/login/lock screen after the computer has gone to sleep. But I kept searching, and stumbled across a notice from Crucial about a firmware update for my SSD, which included a note about resolving issues when the hard drive is powered back up after being spun down or put to sleep. I didn't think it would apply - after all, I had changed the System Preferences to disable ever powering down the hard drives, so that shouldn't be possible. I decided to give the firmware update a try, though, since it would save me from having to disassemble my iMac!

Crucial doesn't have a firmware updater for the Mac, so I downloaded the "Manual Boot File" firmware update package from the Crucial m4 firmware update site. I burned it to a DVD, and then rebooted my iMac, holding down the Option key on the keyboard to force it to allow me to select what drive or disc to boot from. The Crucial firmware updater appeared as a CD labeled 'Windows'. I selected that, selected yes to allow the updater to continue, and then crossed my fingers. Once the firmware update completed, I rebooted my iMac, and much to my surprise, the firmware update completely resolved my issue!

So, if you have an iMac that you've installed a Crucial m4 SSD in - update the firmware! And don't assume that the problem isn't related to powering the hard drive down or putting the hard drive to sleep, because apparently even if you unselect this option from the Energy Saver preferences in System Preferences, OS X is still trying to put your SSD into a lower power mode.

Spotify Killed the Radio Star

I love music - even when I was young, I almost always had headphones on and have over the years developed an interest in almost every genre of music out there. So, when Spotify, a music streaming website, finally made it to the US, I signed up right away! Spotify lets you use your web browser to search for an artist or track and listen to that track right away - free. Or, you can upgrade to an inexpensive paid membership and listen to music from your iPhone or Android smartphone and store music on your computer so that you can listen even when you're not connected to the Internet (perfect for laptops). I don't even both turning on the radio in my car anymore, and I'm tempted to cancel our XM subscription now, too, and save quite a bit of money because Spotify gives me exactly the music I want, when I want it, with no commercials.

The only downside to Spotify for me so far has been the fact that unlike sites like Pandora, Spotify doesn't suggest music you might like or build playlists for you. You have to know what you want to listen to and create a playlist or manually select the next song to play as each song ends. However, I've discovered two great services that make up for this shortcoming - ShareMyPlaylists.com and EchoFi. ShareMyPlaylists.com lets anyone build a Spotify playlist around a theme and then share it so that you can click on the playlist on ShareMyPlaylists.com and have it immediately appear in your Spotify account. So, have a particular artist you like? Want to listen to this week's top 40? Huge fan of 80's music like I am? There's a playlist for that!

EchoFi, on the other hand, works more like Pandora.com - you enter an artist you like and it uses Spotify to play songs that it thinks you will like, allowing you to mark songs you do or don't like so that it can fine-tune its recommendations to your tastes.

I have a number of Spotify invites left, so if you're interested in Spotify, let me know and I'll send you an invite!