Enhancing organization and maintenance of big data with Apache Solr in IBM WebSphere Commerce deployments

Abstract

The main objective of this thesis was to enhance the organization and maintenance of big data with Apache Solr in IBM WebSphere Commerce deployments. This objective can be split into several subtasks: reorganization of data, fast and optimised exporting and importing, efficient update and cleanup operations. E-Commerce is a fast growing and frequently changing environment. There is a constant flow of data that is rapidly growing larger and larger every day which is becoming an serious problem in the current process of data handling. Apache Solr is an enterprise search platform used by IBM WebSphereCommerce. It is a fast indexing engine that can handle large amounts of data with proper configuration, data organization and custom extensions. This thesis results in extensions for Apache Solr programmed in Solr's Java API. Other expected results of this thesis are data organization rules and recommendations that can help with the current big data situation with Apache Solr in IBM WebSphere Commerce. Testing and evaluation of the results are an important part of this thesis and tests are run with currently deployed IBM WebSphere Commerce systems and data that comes from these eCommerce systems.