In the last two years there have been a series of mergers and acquisitions that have completely changed the landscape of search vendors.  This only goes to re-enforce the point I made previously that search in retail is commoditized and reached a level at which innovation in functionality is left to smaller iterative improvements, and the future for search is to not as a stand alone service but as an integral component of a larger vision for ecommerce. 

But how have all these Acquisitions played out and what can we expect from retail search in the future?

• Fast was the first to be acquired by Microsoft back in 2008 and since then there has seemed little focus on retail and more focus on SharePoint integration.
• Mercado was acquired by Ominture which made sense in the pursuit of closing the loop for personalization and merchandising.  However, Omniture was recently acquired by Adobe and my discussions with Omniture now leave me unclear to what their actual roadmap and strategy is.  It seems that the Ominiture product set will remain as a suite of loosely coupled products (e.g. Mercado, sitecatalyst). Adobe has it all the components necessary to create an extremely interesting story for retailers around personalization, merchandising and search.  However, at the moment this story is left to the imagination of retailers and system integrators.  What is needed is a clear strategy and roadmap for all their commerce related products such as Mercado, Omniture and Scene 7.
• Fred Hopper’s price point and licensing model has enabled it to gain a lot of traction in the SME commerce market, however, it was also acquired very recently by SDL who also own the Tridion CMS. SDL PR and marketing are boasting that the acquisition of Fred Hopper adds targeting and marketing to their portfolio with SDL releasing a Fred Hopper and Tridion integration called SmartTarget in Q2 this year.  This acquisition may allow SDL to sell Fred Hopper into other markets and enable SDL to enter the ecommerce and retail market, but whether the combination of Tridion and Fred Hopper will provide any additional value to retailers or whether there will be any new innovations remains to be seen.
• Autonomy, which is rarely considered for retail search also acquired the Interwoven CMS.  This fits well with its overall strengths in unstructured content and I have seen no further indication that they will focus any further attention to commerce and retail.
• Endeca is now the last man standing in terms of big pure play search.  Endeca competitors seem to be suffering from a loss of focus and pace which is usual following any M&A activity.  This along with recent developments including page builder and the SEM modules has placed it above its competitors in commerce and retail search.  Their recent strategic alliance with the Hybris who provides an eCommerce and Product Information Management (PIM) platform show that Edenca continues to invest in commerce and retail.   The interesting thing about the strategic alliance is that it is not just about marketing, in fact Endeca and Hybris have created a joint development team working at integrating the two platforms.  If done right this could bring some exciting new developments across all search and eCommerce functionality.  It will also be interesting to see how they deal with the usual clash over which system will control the front-end and user experience.

Search plays an important role in the positioning of products to customers. Whether helping customers find the specific products they are looking for or to simply browse a product catalogue, search is pivotal in converting customers from browsers to buyers. It has been these obvious benefits of using search for commerce which has driven specializations in search technology aimed at influencing customer buying behavior.

Enterprise search engines such as Endeca have maintained a strong presence in retail and commerce due to their ability to rapidly access vast repositories of data and provide robust architectures that can scale easily to cope with the demands of high volume transactional websites.  The ambitions of enterprise search for ubiquitous information access and the empowerment of information discovery has seen the arrival of more intuitive methods for exploring information that are particular valuable in retail.  Endeca has been the driving force behind faceted navigation a method which enables multiple navigational paths to any one item of information.  The use of facets to support the exploration of large product catalogues was a key differentiator in retail in the past; however, faceted navigation is now considered a de facto standard within most search applications.

Following the commoditization of faceted search vendors switched their focus to merchandising and product placement.  Enterprise vendors have been reinforcing their position in commerce through the refinement of merchandising functionality which is often consolidated into modules specifically targeted at supporting retailers.  These modules give the ability for retailers to populate areas of a page with products based on rule based queries and are fully integrated into the base platform and can be added relatively easily.  The Fast search engine uses merchandising modules for creating the usual merchandising zones while Endeca have taken things a step further by extending their merchandising workbench to allow business users to take complete control over a pages dynamic content and layout using their page builder extension.  This also gives the ability for websites to create dynamic landing pages based on any search term to providing specific merchandising landing pages for brands, categories and click through from paid for search. 

There have been number of retail specific search engines such as Mercardo and Fred Hopper which have concentrated solely on commerce and retailers.  The relative advantage of these retail specific search engines is the depth of knowledge and expertise that comes from specializing in a specific sector. This is made apparent in the maturity of their business tools and interfaces which are both intuitive and highly relevant for retail business users.  Mercado for instance embeds merchandising into the very core of its platform.  Allowing business users to have complete control of where and when products are placed, whether in search results, facets or in targeted ads.  

The question is what is the future for search vendors in retail and commerce? 

I believe the answer does not lie in iteratively adding more bits of functionality to a search platform but in how a search engine plays an important part in the whole solution for commerce.  Search engines are no longer needed as point solutions to address specific problems such as navigating complex catalogues and simple product targeting.  Instead, retailers are craving higher level solutions for commerce strategies to drive and manage their businesses. 

It is in delivering these larger commerce strategies where retail search engines should start to focus their efforts, However this requires them to integrate more closely with of an increasingly complex ecosystem of ecommerce technologies ranging from analytics, eCRM, ecommerce platforms, search, personalization and multi variant testing tools not to mention the wider role of backend technologies ERP, fulfillment, finance and buying systems in supporting these strategies.  Typical examples of commerce strategies where retail search can play a crucial role include closed loop and personalised merchandising in which the targeting of products is constantly refined and influenced by customer behavior.   

For these strategies to be effective retail search engines needs to incorporate new sources of data in their indexes and interact in new ways with other systems in order to provide enhanced customer experiences and sophisticated tooling that businesses need.

I recently volunteered to review several website compliance tools after a colleague’s frustration in trying to differentiate them. When you start searching for these tools you soon find a very crowded market place. However, I was fortunate to be constrained to review just three of the leading vendors, Sitemorse, Magus ActiveStandards and Vamosa Check and Fix. I will summarise what I found by answering some simple questions.

What do website compliance tools do?

Website compliance tools are similar to search engines in that they crawl through the pages on a site or sites based on URLs you supply. As they crawl through sites they analyse pages, content, images, documents, digital assets and code by running a series of predefined tests. The results of these tests are provided as reports highlighting defects such as broken links or failure to comply with certain standards and guidelines such as W3C.

How do they do it?

All these applications operate on two basic principles. First of all they are delivered through Software as a Service (SaaS) platforms and second they work by crawling through a website from a given URL extracting pages, content and digital assets ready for testing.

The question you may ask is why do I need a web compliance tool and what can they do for me?

• Improve code quality and highlight any possible compliance issues during the development and testing phases.
• Provide audits of your web properties across your web estate
• Site optimisation to improve site integrity and areas such as SEO
• Brand governance by ensuring messaging, content and assets are consistent and meet company/corporate guidelines across your web estate.
• Content governance, constancy and quality can be achieved from tests that range from identifying spelling and grammatical errors to tests which monitor key words, phrases, places and corporate terms.
• Compliance to web standards and guidelines such as accessibility, usability and other W3C guidelines.
• Ensure that websites adhere to the relevant laws and regulations

What are the tools strengths and differentiators?

Magus has a very flexible interface built around a dashboard concept. The dashboard provides summary information across multiple sites as well as allowing users to drill down into individual sites and issues. Magus also provides strong workflow to support the assignment and management of issues raised in the test reports and again visibility of assigned issues and tasks are visible from the dashboard. There is a range of predefined tests but in instances where these tests do not fully satisfy the requirements new tests can be constructed by Magus developers. This is particularly useful for non standard tests to support brand and editorial guidelines. Magus also provides a feature called ‘aboutness’ which automatically identifies entities in content such names and companies. This enables a user to find where these entities are used and navigate the reports based on these entities.

The Sitemorse approach cares less about customisation and more about simplicity and rapid deployment. Sitemorse uses an extensive series of tests to create a set of standard reports for all its customers. The fact that all reports are standardised and use the same tests allows Sitemorse to summarise compliance report data to create sector based leader boards. These leader boards are also available for customers to rank there own sites, however, what is not understood is the algorithm used to for generating leader boards and its validity or relevance to particular sectors.

Vamosa originates from content migration with products aimed at analysing, cleansing and migrating content between content sources and management systems. The Check and Fix (C&F) product is a natural extension for Vamosa adding compliance to their solution suite. The mindset of configurability and flexibility required for content migration is firmly set into the C&F product design. C&F is more of a web based framework for displaying compliance reports that are produced from tests that originate from policies that the customer constructs. These policies can also be integrated with content management systems to provide a level of automated fixing. There are a number of preconfigured policies as you would expect for areas such as HTML, SEO and spellings, however, this level of flexibility comes at a price as time is needed to analyse, design and implement policies. As with all other web compliance systems C&F captures content and assets, however, Vamosa also retains different versions of this content allowing you to view content and assets through a timeline.

In a nutshell

Sitemorse is a quick and easy way to audit website compliance issues. Very useful if you need to monitor or test website compliance and have little time to invest for customising the reports or the tests do not require customisation.

Magus is ideally suited for those who need sophisticated governance for large web estates, require issues to be managed through organised workflow or have a need to closely manage brand compliance issues.

Vamosa Check and Fix is suited to providing compliance where complete flexibility is required for creating appropriate tests for complex environments. Also C&F is an obvious choice to support content migrations that are already using the rest of the Vamosa tool set.

Following the post “The implications of the EU Cookie Law” I thought I would clarify what impact there could be for online businesses and users who use cookies. But first we need to review first principles:

1. HTTP is stateless, a website does not know who you are from a simple browser request
2. A session can be maintained by a website where it can recognise a user for a period of time or during a series of transactions by using either encoded URLs or session based cookies that are stored in temporary memory.  In this scenario only an ID related to a session is stored with no user specific details.
3. By using a persistent cookie a user can be recognised across different sessions even if the browser has been closed.  These cookies hold personal details about the user stored in text files on the user’s’ machine.
4. There are two types of persistent cookies, first party cookies given directly from the site you are accessing and third party cookies given by third party services used by the site you are accessing.

So if we assume the EU Cookie law pertains to the consent of persistent cookies, how would this actually effect websites, marketing and user experience? Well let’s look at the broad areas of functionality persistent cookies support:

• “Remember me” and silent log–ins.  We have all become used to not having to remember every username and password as we flitter from Facebook, to hotmail, and twitter.  Compliance for these types of cookies should be relatively easy as consent is given for “Remember me” during log-in, all that is needed to meet the requirement for comprehensive information is a link at log-in to the privacy statement. 
• Site analytics – Cookies are used by site analysis systems(Google analytics, Ominiture, WebTrends, etc) to track a customer’s behaviour while browsing a website to provide behaviour profiles and insights into areas such as drop-outs during checkout, product conversion and campaign effectiveness.  The issue here is statistical data will be inconsistent and incomplete if large numbers of users opt out as they don’t see any value in accepting these cookies.  Site analytics can still track general traffic statistics but these often prove to be to generic or technical to be off real use to the business users.
• Personalised recommendations and content – Product recommendations are becoming more sophisticated going beyond simple logic based on previous transactions and popularity.  There are now dedicated technologies which can track an individual customer’s interactions with a website and recommend products based on their patterns of behaviour. Many of these technologies such as Coremetrics rely on cookies to identify a customer as they interact with a website and some services such as Aggregate Knowledge capture details from across the sites they are active on. Without persistent cookies these systems are incapable of providing personalized recommendations, however, there are services such as Certona and Baynote that can provide personalized content without the use of cookies.
• Multi-Variant-Testing (MVT), Targeting and segmentation – this is similar to personalised content but works on the concept of targeting different versions of content to customers or customer segments and then testing and analysing its effectiveness e.g.. Omniture Test & Target.  MVT is used for simple actions such as optimising home/landing pages to defining complex merchandising strategies and promotions.  These tools often rely on persistent cookies to track the effect of targeted content on users and allow further targeted content to be optimised.
• Targeted Marketing – Sites with display advertising (banner ads etc) will almost certainly drop third party cookies on a user’s machine from marketing networks ad servers.  These cookies are used by the marketing companies to track and target users during any time spent on sites which uses the same ad server. 

There are other methods that can be used for simple tracking, however, the reasons why cookies are the most commonly method for tracking is they enable:

• A specific banner or link that the web visitor clicked to be tracked
• Tracking of users browsing behaviour across multiple sites while the cookie is active
• The tracking of conversion for the period of time the cookie is active
• The tracking of repeated sales
• Ads to “re-target” products a user has viewed on pervious visits to eCommerce websites.
• Ads to be personalized based on a user behavior profile

The use of cookies by marketing companies is probably the most contentious.  Many of the cookies used are identified as malicious cookies by anit-virus and spyware sites for example the revsci cookie is often represented as a malicious cookie.  spywareremove.com 

say this cookie comes from porn and gambling sites.  However, this cookie is distributed by Revenue Science through websites such as guardian.co.uk, foxnews.com and timesonline.co.uk and is used for behavioral targeted of ads.  Revenue Science even quotes a Forrester report on their homepage stating that an Independent Study Finds that Online Shoppers are More Receptive to Behaviorally Targeted Ads than Contextual Ads.   Even if we are more receptive to targeted ads we may not necessarily want to consent to our behavior being tracked by cookies. 

The argument to whether users consent to having targeted Ads or any other service using cookies will depend upon the messaging and how the information for cookies is presented to users.

Despite the very quiet arrival of what is being called the EU Cookie law (see Amendment to Article 5(3) October 2009 Page 77), the debate about the impacts of this small Amendment are just beginning. The vagueness about the implementation of this law does not help and if taken literally the law could move the development of the internet back 10 years, dramatically effect digital marketing and fundamentally change the user experience of any website in Europe. The law is aimed at “controlling those instances where information stored on a user’s equipment” involves “unwarranted intrusion into the private sphere (such as spyware or viruses)”, which is great and everyone welcomes this.

The new Article 5(3) amendment states that information can be stored and accessed “on condition that the subscriber or user concerned has given his or her consent, having been provided with clear and comprehensive information” and according to the amendment each member state including the UK is left to decide how this law should be implemented and enforced

Although the law does not suggest the elimination of cookies a lot depends upon the interpretation and implementation of the law by the EU member states. If taken literally and implemented badly it could mean annoying pop-ups constantly which constantly nag you. Try setting your privacy setting in IE to prompt you for every cookie and see how annoying this can be. This will inevitably lead to switching off cookies all together or accepting everything. A better but not perfect interpretation could mean displaying a privacy landing page before a user reaches any part of the site that requires cookies. This would still have implications for customer conversation in ecommerce and digital marketing activities.

The Internet Advertising Bureau (IAB) UK has a more optimistic view (Consumers given power over cookies) believing this provides a legal basis for cookie management tools in browsers and applications. Browsers today do have cookie management tools as well as anti-virus and security software; however, these are not user friendly or easy to understand for most non-technical users. The real problem is the information supplied by cookies. At present the information describing cookies in a browser based solution is extremely poor and to meet the requirements for comprehensive information much more needs to be done in providing additional useful information. At the moment to find any information about a cookie requires first finding its name from the browser or temporary internet files folder and then searching the web, assuming you know what you are looking for. A browser based solution is going to require more than adding functionality to a browser, it requires thought into

• What cookie information is required by a user to make an informed decision?
• Who is responsible for providing this information and how is it regulated?
• Where does this information come from and how is it stored and managed?
• What is its format/definition and how is it delivered to the browser or other application?

This seems to indicate a more far reaching solution requiring the definition and adoption of new standards for cookie files, formats and protocols, no small task and the impact of which is not be limited to the EU.

I wrote the The effect the EU cookie law has on websites, marketing and users  to provide a more detailed view on how cookies are used to support website functionality.

I thought I would start this blog with a summary of my latest thoughts around eCommerce platforms which I presented at eCommerce Expo recently.  In this presentation I make the analogy between the journey a business takes when it comes to eCommerce platforms to that of a person navigating the property ladder. 

When I first moved to London from the North East to start my IT career in the role of an analyst programmer for Reuters, I was unsure about how it would turn out and after university I had little financial support or security.  I actually slept on a friend’s floor for the first few months until I got myself established. I then rented my first place in West Kensington, a very small studio flat located fairly close to the office.  This suited my (very) basic personal needs (tiny place to sleep, shower, some personal space, cooking facilities and close to work) and matched what I could afford at the time.   As it turned out I had some great times there and fortunately my career soon flourished.  

This is somewhat similar to a fledgling start-up ecommerce businesses whether a completely new business or an existing retail brand goes online – they are often uncertain as to their future revenues, have basic requirements (home/landing pages, browse, product details, basket, checkout, payments, etc) and their budgets for development are very limited.  In this case, businesses usually opt to try and build their own site from some open source framework, use a low cost service such as eBay or engage a SaaS based eCommerce provider.  These options simply get you up and running with the bare minimum that you need to sell products online.

Once I became more established at work and socially connected in London I started to outgrow the studio.  I desperately needed more functional space to live in. I had no room and so much stuff.  Having a shower, kitchenette and bed in the same small room started to loose it quaintness and it was not a place where I could invite my friends for dinner.  At this time I was still not in a position to buy especially at the crazy property prices in London, so I rented a slightly larger flat. It was still quite restrictive in space but met my immediate needs perfectly.   

 This is what I have encountered time and time again in online retailers reach a point in which they outgrow their start-up platform and look to re-platform as they require more functionality on the site, better tools to manage the business and increased scale.  Often these businesses are not yet in a position to buy, implement or even manage a fully fledged enterprise ecommerce platform so they move to SaaS platforms such as Snow Valley or Fresca. 

Alternatively, they may try to augment their home-grown platform with specialist technologies to address specific pieces of functionality and resolve particular technology issues.The most common areas I’ve been involved with in these types of situations are:

• Improving search, browse and merchandising using software like Endeca
• Improving content management by integrating with a CMS (e.g. Fatwire, Episerver)
• Adding mediated product ratings and reviews(Bazaarvoice, Buzzilions)
• Adding product recommendations Cetona and Baynote
• Managing product information and catalogues (Hybris PIM)

The next stage in many people’s lives’ is they meet someone live together and get married.  If you are still in your comfortable bachelor pad you may find that you need more flexibility in terms of décor and interior design as well as more space for the masses of shoes you’ll no doubt inherit.  I think a whole blog post could be devoted to women, their shoes and a man’s inability to understand why. At this time you will start to look for a new property with a whole set of new requirements you wouldn’t have considered before such as the size for a new family and local schools.  Fortunately for me being a technical strategist at heart I had already planned ahead and bought a house before I was married with plenty of capacity, good location for schools, transport links and neutral decoration to act as a blank canvas for my wife’s decorative taste, and yes I had space for those all important shoes.  

Again I have seen many businesses reach a certain point where the SaaS based services seem to become too restrictive for the business.  Despite the claims of complete flexibility, SaaS based providers have the problem of trying to balance the flexibility for customisation and the benefits it gets from running a generalised platform e.g. costs, maintenance support, etc.  Once the ecommerce business reaches a point in which their requirements go beyond the stock features and functionality of the SaaS platform, custom development is required. As development is not the core competency of many SaaS providers, businesses will have to contend and wait for development resources to become free. 

Business control can also become an issue as a business matures – an example I have seen is an online business trying to extend its range of offerings to use new third party suppliers which conflicted with the SaaS provider’s business model.  Although, nothing is ever impossible the outcome usually results in escalating costs, slow response times and friction between the business and the SaaS provider.  Furthermore any new innovative developments in functionality are often shared with the rest of the community making it difficult for mature businesses to define and differentiate themselves.

When an online business is in this situation they start to look at ownership models using licensed platforms and regardless of which supplier they still have the option of a self managed or fully hosted solution depending upon their own internal IT capability and appetite.  At this stage they will find themselves just like a newly wed family searching for a house, looking at a whole new set of requirements they couldn’t have possibly imagined as the technology supplier spins the web of new features, possibilities, platform considerations, integration technologies.  Just like selecting a mobile phone they all seem to be completely essential. 

One of the big problems is that if your are moving from a SaaS platform, all that lovely work you did is lost as SaaS based platforms tend to be closed technologies.  There are also a series of  clandestine meetings conducted with suppliers as the business is afraid of the repercussions if the current supplier finds out about their clever plans to move.   A business, soon finds out that re-platforming on the enterprise system is a huge undertaking taking months of development with the total costs for licensing, infrastructure, and services reaching as high £5 -10 million.

Each move a business makes to a new platform comes at a massive risk and cost.  It takes the focus away from what is core (to SELL) to months of strategy, selections, requirements, development, testing rollout……..

My own experience and talking to my friends in the industry, there seems to be a dizzying number of businesses that   re-platform every 2-3 years.  However, I also have clients who have huge online businesses running on bespoke platforms which are more then 6 years old – although they creaking a bit right now.  I guess the moral of the tale is to be cautious when deciding what platform to choose because what may seem right for today is not necessarily right for tomorrow.  I was always told to stretch to buy the best house I could afford and satisfy the needs over the long run, rather than suffer from the inconvenience and cost of constantly moving.

Video of my presentation for ecommerce Expo http://www.seminarstreams.com/ecom-expo-2.asp