Another year has come and gone since we last prognosticated about the key Enterprise Search players, see Transatlantic Titans Square Off in 2006. In this issue we take an updated look at the past year and what 2007 may hold; not too many fancy predictions -- that's not our style -- but at least a preview of what you'll be seeing.

Key Players Still In Front

FAST and Autonomy Still Tier 1

Autonomy and FAST Search continue to be the dominating players, with many other players large and small nipping at their heels.  Both companies have done well according to both their PR departments and most of their customers.  Autonomy was still working through the merging of the Verity products and staff retention issues from their acquisition of Verity in late 2005, so it felt like FAST had a bit of a "smoother" year, but both companies remain the two 800 pound gorillas in the room.

Both FAST and Autonomy have moved favorably in "Gartner's Magic Quadrants", in the past few years, with FAST now essentially tied with Autonomy after FAST's rapid ascent. The "At Verity" represents Autonomy, which acquired Verity in late 2005, and has therefore subsumed Verity's spot the 2006 data point. Endeca is a dotted-line only because our focus here is on Enterprise search. We wonder if Gartner will move Google and IBM into the top right quadrant this year as well.

FAST: Still the "Bring It On!" Guys

As a company, FAST Search and Transfer has shown no fear of any potential search application, no matter how large, no matter how complex, no matter how unusual the market or requirements; if you have the money they can build it, period.  FAST takes the approach that search is a highly flexible platform on which other corporate applications can be built.

Of particular note, FAST has had traction with their eCommerce solution called FAST ImPulse, a direct response to Endeca's core product offering.

FAST also provides a number of customizable vertical applications they call "Search Derivative Applications" (SDAs) in widespread fields such as law enforcement, newspaper and media management, as well as directory and yellow pages software. However, for extensions not directly tied to search, FAST has taken the approach to partner with companies that provide best-of-breed technologies in audio and video search, compliance, and other specialty areas.

FAST Forward '07, which just finished up in San Diego, continued to showcase this with lots of new ideas and demos.  They are establishing their annual user group meeting as a bona fide industry event, with speakers like Andrew McAfee, John Battelle, Tim O'Reilly and others.  We look forward to the introduction of ESP 5.1 later this year.

Autonomy: the "Ain't it Cool!?" Guys

Autonomy has had some positive traction with their "K2 v7" product, a hybrid of their core IDOL product and the Verity K2 API.  As a reminder, Autonomy now has 3 different search engines via acquisition: its own IDOL core, the Verity K2 wrapper around IDOL, and Ultraseek.

K2 and Ultraseek are both contributing to the evolution of the IDOL product line; Ultraseek donated some excellent spider technology, and K2 provided a more robust API, toolset, user, and integration layer.  IDOL already had "whiz-bang" core search and was already quite scalable.

Ultraseek, the third code-base in Autonomy's repertoire, is still beloved by customers, but starting to show its age. It's still a great product for "search dial-tone", we still love it, but we don't expect many "Enterprise Search 2.0" enhancements for it.  While two of its core engineers have left, three still remain and the product is still being actively maintained and incrementally enhanced.

If you're still using K2 v6 we do encourage you to start plotting your migration path forward.  Many generic K2 users will probably migrate directly to the core IDOL product.  However, if you are a K2 user and have invested heavily in scripts and management processes, you might find that K2V7, also based on the IDOL kernel, will help protect the investment.

Autonomy has also taken the approach of offering "total solutions" in search: aungate provides Email scanning for compliance, Virage includes multimedia search for audio and video content, etalk for CRM, and Cardiff for Business Process management.

The Challengers

Two of the four US Behemoths from last year's roundup have started to cross over, trying to compete with Tier 1 players.  Google and IBM have both made great strides; we still consider Oracle and Microsoft to be Tier 2 players.

A Mixed Year for Google

If you look at the numbers, Google had a good year.  Business Week had reported a doubling in sales and certainly every client we've talked to has taken a peek.  However, they failed to meet the functional requirement at a number of test installations.  By our tracking, Google has lost business to our Tier 1 companies much more often then they succeeded.  Many clients have tried the GSA with high hopes, only to later abandon it and then buy yet-another engine; this is not universal and they certainly do have some happy enterprise clients.  Certainly every company we've talked to has had a high-level executive ask "Why don't we just use Google?" somewhere along the process; they then try it and have an answer.

Google has released new versions of their product and are fighting back really hard - every VP in America expects them to "fix" enterprise search and they know it.   They've also tried to address integration problems by announcing high-visibility partnerships with companies like Bearing Point, but apparently not an exclusive deal given the amount of spooning Bearing Point has been doing with FAST and potentially other vendors.  Google's curse may be their own brand name.  Folks have such high expectations: "Finally our enterprise search will be wonderful!"  But inside the firewall, deprived of their link ranking data, they perform on a par with many other "search 1.0" engines, enabling competent "search dial-tone", but not the magic relevancy that folks expect.  Meanwhile Ultraseek and IBM's new low-end offering with Yahoo, often provide comparable performance at a lower price.

IBM: The Biggest Positive Surprise of 2006

By far the biggest surprise this year was from IBM, with their OmniFind Discovery Edition engine.  As you recall, last year we had counted them as one of the four US behemoths that was still not in a leadership position.

However, in 2005 IBM acquired iPhrase and in 2006 re-released that product as IBM OmniFind Discovery Edition. It is also available in a reduced feature set version as the Yahoo! Edition we'll talk about shortly.  We're impressed with this new commercial offering, and we see it as a direct threat to Autonomy and FAST in the mid to long term.

IBM's new commercial offering now is a solid Enterprise Search 2.0 product, with full results list navigation, advanced linguistics, excellent business logic, and of course the ultimate blue chip brand name.  It still faces some technical and marketing challenges.  The current version may not scale as easily, or integrate with other systems as completely as FAST or Autonomy, but IBM certainly realizes this and is busily working to address all of these issues.

Of note, Discovery Edition offers a powerful platform for custom business logic to handle specific searches.  For example, employee searches related to vacation and sick time could automatically include a custom applet, embedded within the results list along with the organic results, to help them schedule their leave.  Sure all engines are capable of this at the application layer, but IBM has thought this through much more thoroughly than anyone else we've seen, and the framework will make this a much more refined and common practice.  IBM has other interesting tools up its sleeve that are tangential to search and, if fully integrated, could make for an even more compelling solution.

And of course clouding up the low end of the market was the release of their other search product, the free "IBM OmniFind Yahoo! Edition", which supports up to 500,000 documents, and includes an admin GUI and support for custom synonyms.  Look for a review in an upcoming issue.  Don't be confused -- the Yahoo! Edition is not a same as the high end OmniFind Discover Edition, but it is a direct shot at Google's lower end appliances, and many of the Tier 2 / Tier 3 players.

Endeca tops in eCommerce, but now an Enterprise Solution?

We are impressed with Endeca's technology and dominance in the eCommerce space.  They were one of the early leaders in faceted navigation, and the first to tie dynamic business logic to search relevancy.  If you are designing an eCommerce solution, then we believe Endeca and FAST's ImPulse should be on your short list.  And of course they are now the only US-lead company in Gartner's "magic quadrant".

But in 2006 we saw Endeca talking more about enterprise search, and we're not sold on it yet.

Forrester helped to validate this claim with a favorable evaluation of their enterprise search administrative tools, an area that is lacking in some other vendors' offerings.

But we can't help wonder, since they dominate the eCommerce space, why not be happy with that and continue to grow the lead?  We imagine the answer is that, to sustain growth they want to expand into other markets; but is the eCommerce market now flat?  Even if the eCommerce markets in the US and Europe were assumed to be saturated, there's still the rapidly expanding economies of Kramer's "BRIC" (Brazil, Russia, India and China)

We think they are very late to the enterprise game; they are very rarely mentioned by clients outside of the eCommerce space.  Many other larger players are already well established or are spending big to get there, so we are skeptical of them in this market.  Do they seriously want to take on Autonomy, FAST, Google, IBM, Microsoft and Oracle?  We think their energy would be better spent localizing and marketing their excellent products in countries that are still at that start of their eCommerce "boom".

Still Not There, but still trying!

US Behemoths in General

As we mentioned last year, if you compare the market cap of the European Tier 1 players, compared to the 4 US behemoths trying unseat them, it really seems like a David and Goliath battle.  If you compare Google, IBM, Microsoft and Oracle to FAST and Autonomy, it's on the order of a 300 to 1 market cap ratio.  But for those large players, enterprise search is not their main focus.  If any of them stopped selling enterprise search tomorrow, it wouldn't have too much of an impact on their bottom line.

Even for Google, which has evolved into an advertising company that happens to have a famous public search engine and portal, enterprise search just isn't a "must win" segment for them.  To be fair, it is believed that Google's GSA business is the largest non-public-portal revenue generating product line they have, though still not "core" to their bottom line.

Microsoft Planning a Big Push

Microsoft continues to promote search within the enterprise, but we still list them as a Tier 2 player.  Vista has shipped with search that is more deeply embedded, but we still see this as "desktop search", competing with the Google Toolbar and Apple's Spot Light.

Microsoft has gained some traction with SharePoint, and the search that goes along with it; but in our minds this is still "search dial-tone" / search 1.0 level search tied directly to SharePoint.  We'll wait to see if SharePoint 2007 for Enterprise Search starts "fanning out" to other repositories and applications.  Microsoft also offers search in many of their other server products, including SQL Server.  But like the other US behemoths, full-text search is not their primary product, so by definition is not their top priority.

Oracle taking aim at Microsoft via Acquisition

Oracle already had a search product, their Oracle Text.  But in 2006 they acquired Stellent, taking direct aim at Microsoft SharePoint.

We're still not seeing heavy traction for Oracle in the full-text search space.  Adding Stellent will help with the CMS market, but that is still a somewhat large niche.

All the other Players

The Tier 3 players are a very heterogeneous group, with some players that were former leaders who have not kept up with technology; relative newcomers with solid technologies like DieselPoint, iSYS, and YourAmigo; giants with long established reputations that seem to maintain a low profile like Open Text; and may others with more vertical focus like X1 and Recommind.  We've briefly mentioned these in a sidebar above, and will be following up in future issues focusing on each of these vendors.