Bigdaddy was a turning point for hardcore Google organic SEO. Why? Graywolf said it best:
What I am here to say is I think we are hitting the leading edge of a
new force to be addressed with, which for lack of a better term Iâm
calling âsandbox crawlingâ. Hereâs the way I see it, if your website is
missing the right âquality indicatorsâ what youâll start to see is
superficial crawling and indexing of your website. Your site which may
have had hundreds, thousands or even hundreds of thousands of pages
will just not be as well represented in Googleâs index as you would
like it to be.
Great, now we have to worry about ‘just getting indexed’ again? What is this, 1999?
Matt Cutts of course had already basically confirmed what Graywolf said:
Some folks that were doing a lot of reciprocal links might see less
crawling. If your site has very few links where youâd be on the fringe
of the crawl, then itâs relatively normal that changes in the crawl may
change how much of your site we crawl. And if youâve got an affiliate
site, it makes sense to think about the amount of value-add that your
site provides; you want to provide a reason why users would prefer your
site.
Of course it always comes back to links, and link = trust. (And now for you affiliates, you have to make sure you have a ‘value add’ in Google’s eyes, or the Eval team might nail you.)
But back to indexing. Domains with very low trust will have trouble getting fully indexed (they may go home-page only, or have, say 10% of their pages in). Now, for a domain that gets fully indexed, we won’t know how much trust it has–it could be "very trusted", or "mostly trusted", or just "somewhat trusted"–but we will know it at last has more than "very low trust".
Which in my opinion is very useful. Aside from whether a site ranks or not, this is the only indication Google is going to give us on any given domain’s TrustRank. PageRank they leave wide open, but obviously TrustRank is now the trump card to rank. Try checking that in the toolbar.
So here’s how observing the sandbox crawling helped me. I recently launched a site — database-driven, affilitiate, and yes value-added with all unique content thank you for asking — on a 2003ish PR5 domain. I expected, of course, to leverage all of its trust to gain instant rankings, wealth, romance, and fame
So we went live with the site, and one week later — still homepage only. (When we acquired the site/domain, it was just a one-pager, so this didn’t represent a decrease in pages indexed.) Two weeks later — homepage only. (By this time we probably had 200 pages or so live.) Three, four weeks later — still homepage only.
Mama didn’t raise no fool, I don’t know why or how, but I know this domain has the proverbial "TR0". Rather than whine and complain about it (well, actually, I probably did that for a bit), we grabbed a new domain — something brandable — and switched the site over.
Now a new domain probably isn’t any better than the other one we had, and I wanted the launch to go right, dang it. So I grabbed it every trusted link that I could (quickly) — Dir.yahoo.com, Sbd.bcentral.com, Business.com, a hosted adverpage on an older domain, and an in-content link from an old, ranking (trusted) related site that a friend owns (Thanks mate!)… Two days later, bam! 28 pages in, four days later, 160 pages in.
What’s today’s lesson? Maybe two things. (1) If you need to determine how much trust G gives a particular domain, watch how quickly Google does (or doesn’t) index a lot of new pages on that domain. (2) When you launch, launch with some nice trusted links, it’ll get you indexed and "somewhat trusted" status, and will start your domain’s "linkage history file" off on a good note.