For example, if you visit this URL, you’ll get a JSON file of results for Harry Potter films:

http://sg.media-imdb.com/suggests/h/harry.json

The “h” directory means the query starts with an “h,” as they group their result sets alphabetically, and the “harry” part is what was typed into the search box. So if you wanted results that would match Doctor Who, you could use /d/doct.json. (Spaces are replaced with underscores.)

They only seem to have result sets for 4-5 character inputs, though. So you can query “ince” but not “inception.” The latter will result in an error. I guess most searches common enough to be matched in the suggestion box are covered within that limitation.

It’s a clever implementation, and it has to save a lot of computing power on a site that large, in addition to being fast.

(Note that this is not a public API, and IMDB/Amazon probably wouldn’t be happy about you scraping it or anything like that. But it’s a nice thing to learn from.)

Unfortunately, Google’s changes have been affecting legit blogs. One noteworthy example is Cult of Mac, a blog that aims to provide “timely news, insightful analysis, helpful how-tos and honest product reviews about Apple and Apple-related products.”

Cult of Mac has experienced the opposite of Google’s goal: their content has largely disappeared from Google’s SERPs, while content farms and spam-blogs scraping Cult of Mac posts have been pushed to the top.

A lot of our traffic came from Google, which is why the changes are so serious. I’m already seeing a big drop-off in traffic. Over the weekend and today, the traffic is half what it normally would be.

I’m pissed because we’ve worked our asses off over the last two years to make this a successful site. Cult of Mac is an independently owned small business. We’re a startup. We have a small but talented team, and I’m the only full timer. We’re busting our chops to produce high-quality, original content on a shoestring budget.

Indeed, Cult of Mac does break a lot of stories. Along with the Boy Genius Report, Mac Rumors and 9 to 5 Mac, they together are the source of the lion’s share of Apple-related reporting. It’s strange that Google’s algorithm would red-flag them as a content farm. Perhaps it is a result of “splogs” scraping their content; maybe a glitch in Google’s secret algorithm is causing one of the spam blogs to be marked as the original source for some reason or another?

There is also an App Store Instant and Instantise, a collection of various instant searches…

Since everyone else is making instant searches, I took an hour or so of my time and whipped-up an Instant Twitter Search for the fun of it.

This video demonstrates it well:

It takes a little time to get used to, but it actually does seem to be quicker. There isn’t a noticeable lag while results load, even on a 1mbps DSL line. The feature does disable itself on slower connections, though.

Google Instant: It Searches Before You Type [TechCrunch]

First of all, slow down. Google can read iframes and JavaScript. Google very likely is already programmatically diferentiating between comments and page content anyway, and tweaking their rankings accordingly. PageRank is not a cut-and-dry “links in minus links out equals rank.” It’s a complex system involving a lot of calculations on Google’s part. I imagine they value links differently depending on what sort of link it is.

Second, stop worrying about it. Google’s business is to provide good results. If everyone tries to cheat the system with a bunch of silly schemes (such as PageRank sculpting and JavaScript links) Google will just change things again. If everyone plays by the rules, and doesn’t worry about their ranking too much, it all works out in the end.

I had been hoping that Microsoft would help create some more competition in the search space, but it doesn’t look like anything has changed.

The design looks virtually the same as the old Live Search (Live.com now redirects to Bing) and the results seem pretty much the same.

It seems like Microsoft just bought a new domain, made a new logo, and drummed-up some publicity. I can’t necessarily know that for a fact, but it seems to be the case to me.

Since launch, the BOSS API has been provided entirely for free. Now Yahoo is putting in place a freemium model where it’ll be free only for developers who generate fewer than 10,000 queries per day. After that, a tiered pricing model will kick in that charges for BOSS as if it were a utility (think AWS). Rates will vary depending on the type of query (web result vs. spelling correction, for example), how many results the developer wants returned per query (with a new maximum of 1000 results), and just how far the developer goes over the free queries cap. (Source: TechCrunch)

It seems reasonable, and Yahoo certainly is improving upon the service. Ten thousand queries for month is a pretty fair ceiling in my opinion. It’s plenty for experimentation or development, or even a web app of reasonably small size.

If Yahoo can leverage BOSS to save themselves from possible bankruptcy, and still have a unique and powerful service, that’s definitely fine by me.

Now pay attention Google: You dumped your API back in 2006, and it’s going to come back to haunt you unless you follow Yahoo’s lead. After a long draught, Yahoo’s getting their game back on, and they’re out to with the developers over with all of their tools.

• Yahoo Readies The Meter For Its First Web Services Business
• Yahoo! BOSS: Now Actually Good for Google

The new service is called “Vertical Lense Technology.” It allows for topical search engines to be created. These search engines, as seen on TechCrunch, the only site to use the service so far, can

• Index the partner site in real-time. Whenever a post or comment is added, it’s sent to Yahoo to be instantly indexed.
• Allow ranking tweaks. When you search on TechCrunch, you are shown a carefully adjusted mix of web and Crunch Network results, ordered in a way that hopefully fits their audience.

TechCrunch describes their experiences with Vertical Lense on their website.

Since we publish using WordPress, supplying our data using the first API essentially required that we design and deploy a plugin that would send information to Yahoo’s servers every time there was a new post or comment on any of our blogs. We also needed to create a similar data indexing system for CrunchBase so that contributions there would show up in the results as well. To ensure that all of our archived content was incorporated in the search index, we supplied Yahoo with historical data dumps from all 10 sites. Perhaps needless to say, this took a considerable amount of time just to ensure that the data we indexed at Yahoo was accurate and complete.

You can read more about this on the Yahoo Search Blog.

Now, when can I get one of these?

What I want to know is what Google thinks about duplicate content cases such as Reference.com or the Associated Press.

Head over to Reference.com, the encyclopedia branch of the Ask.com network of reference sites. Enter a search term. Now go over to Wikipedia and enter the same search term. They’re the same! Reference.com is pulling Wikipedia articles onto their site and throwing in a few ads. (How are they doing this? Does Wikipedia have some sort of API?) What does Google think of this?

Or what about Associated Press articles? They’re syndicated by many newspapers, and appear on their websites. That means the same article on multiple sites, no?

Is Google demoting these pages in their results, or are they giving them a free pass? It’s hard to tell. Reference.com as a whole has a toolbar PageRank of 8, while their iPod article is listed as N/A (while the original Wikipedia article has a PageRank of 7). So that would lead us to believe that they’re being demoted. There’s not really any way to tell for sure though, is there?

It seems that the algorithm is working, and filtering out pages such as those, but I’d like to know what the search giant’s opinion is on such pages. Is duplicate content simply duplicate content?

Wouldn’t it be cool to have something like that on your blog? A search form that, as you type, displays the results in a dropdown instead of a results page? That’s where Sikbox comes in.

Sikbox is a Yahoo BOSS app that allows you to create a cut and paste live search solution. It’s free, easy to install, and you can even apply your own CSS styles to it (or pick from one of the pre-made themes). You can search the entire web with it, or limit it to your website, like most of us would probably do.

Here’s a screenshot of it in action:

If you click on one results, you jump right to it. It works pretty good. It seems to work a lot better than the default WordPress search system too (providing your blog is in the Yahoo index).