We interviewed Mike, an IT Manager at a US company staffed with highly skilled information workers. Because of the competitive advantages that search provides his firm, he's asked that his company name not be used. Mike, a subscriber to this newsletter, was willing to share with us lessons his company learned when integrating Endeca into his company environment, primarily with their content management system from Tridion. We've added links to a number of the products and services references, not as an endorsement but to make it easier for our readers to learn more.

Q: Can you briefly describing your team and how you are organized in your company?

A: I am the VP of IT. I oversee all the IT operations reporting to the CFO. I have a staff of 7 people who have various roles like help desk, service support, and infrastructure development and support. We do have some development and database support; I also have an offshore development team in India who work on our content management, SharePoint (www.microsoft.com/sharepoint) and Endeca (http://www.endeca.com) implementations.

Q: Could you briefly describe a particular project you worked on this past year?

A: We are currently migrating our website from a homegrown content management system to Tridion (www.tridion.com) . The current system was built in early 2000 using ASP and some other outdated applications running on a site server, we are basing the new one on Tridion and integrating it with Endeca. Our new website and content management system provide content to our clients, prospects, and our employees.

Q: What are a couple of lessons you learned from this migration process?

A: We had large amounts of data that we had to migrate from HTML, which was in a SQL Server Database, to XML. Cleaning up the data into pure XML took a far greater amount of time than originally planned. We had a lot of non-compliant XML data. The big lesson was that content is king: you really need to understand what you are getting into. We contracted with Tridion; they said "We can do this, this is pretty straightforward," but it was more of a sales pitch than reality. When the professional services folks got here, they realized that they have bitten off more than they could chew. But they worked with us and we got through it. It took some time, we went back and developed a new plan of how to handle all the data which went back to 1995. Some of our pages were HTML 1.0 with a lot of Microsoft tags. We had to strip out the bad tags and make it XML compliant, which was a project on to itself before we could even implement the content management system.

Q: Your recommendation to other folks looking at doing that is to do a small pilot to flush out the process and understand your data?

A: Yes. During the first major pass through we realized we were going to have a problem. We wound up breaking the content down by years, which we had to do anyway, because we learned there was variation in the HTML code over the years. A conversion script that would work for one year wouldn't work for the next year. We found ourselves going year by year by year to get the content in; we started with the current content and worked backwards to get there. We wanted to make sure we had the most recent content correct, and then we worked back in time to 1995. It was very methodical at that point. Now that it's done, it's in the Tridion system 05.01 and works well within the Endeca application, because it's all XML based.

Q: So, what went really well?

A: Connecting to Endeca went very well. Because Endeca just needs the XML feed, we were able to just plug it right into Tridion and it was good to go. In our old system we were running some scrubbers out of our own system to get into Endeca, and then feeding into the pipeline. Now, the XML content is in Tridion and we don't have to do the scrubbing. So I can say that it has actually made the process much easier.

Q: Looking forward, what significant challenges are you working on now?

A: Well, one of the things we are doing internally is tying all of the information together. About two years ago we built a BI dashboard using the Endeca tool, now we are building on that to create a more complete knowledge portal. We want to use Endeca and combine internal content with external content. We want our employees to access any information they need by going to a simple search box. Because we are research and advisory firm, we have lots of information, but much of its in E-mail.

Q: What is your approach to getting the information out of E-mail into a form that others can access it?

A: Again our approach is to start with current and work backwards. We are trying to get teams to use SharePoint rather than email for current content. We are pushing our organization to be more collaborative in SharePoint using discussion groups, wikis, and blogs. Projects are setup in open files where people can get to the project information and share that information.

Our knowledge portal will have access to internal information as well as 20-30 external sites. We will index all of this content within Endeca which supports drilling down by dimensions. When users do a search, they can drill down by author and find content by that author whether it is internal or external. We will also support drilling down by type of publication, year written, keyword etc. Users no longer need to know where the source of the information is, we will present it to them. The dimensionality of the Endeca application gives you the ability to drive down to get your answer; to find more about that, to drill into that. I want to setup an analyst news related site for them, so that it's very specific to them. The goal would be to cut through all the other clutter and just deliver the information they need, including it with all of the knowledge and information that's already in our research.

Q: Are there any questions you'd like to pose to our readers?

A: I would love to get some tips for how to get the organization to switch from using E-mail as their filing cabinet to SharePoint or collaborative environment designed for that purpose. My department is doing it now, since I have set up SharePoint sites. As we do projects, all project documentation must be up there. I try to practice what I preach. I guess my next step will be to identify other departments with like-minded individuals who are tired of figuring out whose E-mail inbox has the latest document or sorting out software incompatibility issue when trying to open documents.

Going team-by-team is a slow process: I would love to force people to use SharePoint by shrinking their mailbox sizes and folder sizes and locking their desktops! But I think people would cringe at that.

Mike, thank you for your time. Readers, please let us know if you have any suggestions for Mike about how to get people to stop using E-mail inbox as their filing cabinet.