First things first, we selected almost a hundred criteria to get a shortlist. The aim of the shortlist was to help us classify websites (we reviewed almost 400 websites) and determine what site was worth a deeper analyze. We chose to make four tiers with all those criteria, two failures were mandatory and excluded the website from the next tier.
We selected 15 criteria to determine what the website is worth in terms of readability, access to information (key action links from the homepage and clicks to completion), visibility of important links (quick order forms, email, about us, credentials), downloads times, visual quality, registration process and depth of the site.
We selected 12 criteria to rate their reliability. History of the company, credential validation, certifications, accuracy and crosschecking of data, affiliation programs, advertising, web release, forums, blogs, reviews, Bizrate, BBB, Alexa, complaints boards, rip-off boards, and so on. We reviewed everything.
Low price guarantee, consistency of the price across the servers of a game, stability of the prices and competitiveness were the main criteria.
We monitored their payment methods. Credit cards, Paypal, Check and money orders, Authorize.net, Cashu, Wiretransfers, Echecks, Western Union, Google checkout, Moneybookers. We also compared their contact methods, instant messengers in-game pms, emails, live support and phone support (before the trade, during the trade and after the trade) We studied their delivery service. Is the delivery process well documented? Do they list average delivery times? Is it a real 24/7 service? Do they schedule pick up time? Do they provide many trade modes and what trade modes?
We also checked their feature set: FAQ, email and phone support, news, newsletter, special offers, refer a friend programs, affiliation for customers, online forms quality, live support quality, repeat customers program, coupons, phone verification, refunds, cookies, order tracking, currencies, languages, and so on. We checked something like 50 criteria.
We also monitor loads of websites and forums. Alexia, pagerank, bizrate.
We run our own tests.