The Need for Data Mining in the Business World, Dissertation – Literature Example

Abstract

Recent technological developments have created an environment that allows companies to gain information about markets, trends, competitors, and new approaches like no other time before. Data mining is not only beneficial for research and development departments, but any area within the business that is looking to obtain a competitive advantage. While – according to Singh (2012) – the majority of managers think that information overload at the workplace creates extra challenges in he 21^st Century, they admit that information is essential to make decisions. The author will review related literature and research to determine how businesses can create  competitive advantage through data mining, and how they can reduce costs associated with research and data warehousing, while overcoming privacy and ethical challenges associated with the new technology.

Introduction

Data mining has been used by several organizations and private companies for years. Many computer technology firms have developed systems to assist companies with research and analysis of data. While the abundance of information is already creating great challenges for managers, it is possible to use data mining to create competitive advantages for the business. As the technology is relatively new, the number of related studies and research in the field is limited.

Before reviewing studies, research, and related literature, however, it is important to clarify the definition of data mining that the author will use. Pal (2011) states that data mining “refers to the process of analyzing data from different perspectives and summarizing it into useful information by means of a number of analytical tools and techniques, which in turn may be useful to increase the performance of a system” (Pal, 2011, p. 8).  Kumar, Tyagi, & Tyagi (2014, p. 1) however use a more simplified definition, stating that data mining is a “technique for automatically and intelligently extracting information or knowledge from a large amount of data”.

Bal, Bal &  Demirhan (2011, p. 3) state that “data mining is the process of discovering meaningful new correlations, patterns and trends by sifting through large amounts of data stored in repositories and by using pattern recognition technologies as well as statistical and mathematical techniques”.

As the current research is focusing on the capabilities of data mining applications that can be beneficial for the business, the author will use Pal’s (2011) definition when talking about “Data Mining” in general. Further, the study will be based on the assumption that data mining activities are never standalone tools, more importantly a part of a major“ business intelligence or knowledge management initiative” (NASCIO, 2004, p. 1).

Similarly, it is important to set a scope for the research, and identify the business roles that will be examined related to data mining. The current study will focus on uses of data mining facilities that can create competitive advantages in business, such as sales analytics, claims analytics, customer relationship management, and resource management.

Historical Background

Bal, Bal &  Demirhan (2011) review the history of analyzing large amount of data for gaining information that can improve business processes. The authors data the “birth” of data mining to the end of the 1990-s, when technological developments were advanced enough to allow companies to analyze data, and software capabilities matched the demand of the business to gain insights about the market and the trends, as well as internal processes. The authors, however, also state that the main obstacle in front of developing data mining systems at this time was that “capabilities for collecting and storing of all kinds have far outpaced our abilities to analyze, summarize and extract “knowledge” from this data” (Bal, Bal &  Demirhan, 2011, p. 2).

Sharma, Kaur & Manju (2013, p. 696), however, state that data mining has existed since large amount of data could be stored on company computers. However, the business analysis method only took off when three essential technologies were fully developed: ability to collect large amount of data, high speed multiprocessor computer technology, and algorithms that support the collection and analysis of data. The authors determine four different stages of data mining development: data collection that started in the 1960-s, data access supported by relational databases emerging in the 1980-s, data warehousing and decision support in the 1990-s, and data mining processes that are present today.

While the capabilities of today’s data mining software and methods are much higher than they were at previous stages of development, it is important to note that the value of data mining can only be determined by the competitive advantage it helps businesses create in the global marketplace, thanks to the emerging technologies (Al-Azmi, 2013).

Olszak & Ziemba (2007) highlighted one important trend in the development of management information systems over time. The more advanced and intelligent the systems developed, the higher their complexity level they became. While the systems’ capabilities have increased, they became harder to understand and implement. The challenge of the 21^st Century is to develop systems that have a customer-friendly interface, fast processing time, real time analysis and cost-effective software applications.

Technological Background

Several authors have attempted to create a descriptive framework for data mining processes and applications. There are several steps of the knowledge discovery process identified by researchers, such as collecting data, cleansing it, warehousing, data integration, selection of task-relevant data, pattern evaluation, and finally the knowledge discovery.  Kumar, Tyagi, & Tyagi (2011, p, 1) created a graphical representation of the process, which can be useful for researchers and computer specialists to analyze the effectiveness of each step. Table 1 shows how the steps are connected and result in discovery of knowledge that is relevant to the task, business role, and overall objective.

The graph clearly identifies that data mining is the core activity of knowledge discovery process, and without it, companies would not have relevant, checked, and screened data to analyze.

Talking about another aspect of business information systems, Olszak & Ziemba (2007) describe data mining as a part of business intelligence systems. The authors determine the main objective of business intelligence systems as improving competitiveness. The chart created by the authors shows the entire process of turning data into competitive advantages. The process contains one important aspect that other authors do not cover: data needs to be turned into information and knowledge, and support business decisions that improve the business’ competitiveness. (Olszak & Ziemba, 2007, p. 137)

Rygielski, Wang & Yen (2002, p. 487) define some important processes that determine the success of a data mining application, such as discovery, predictive modeling, and forensic analysis. Discovery can result in conditional logic, affinities and associations, and trends and variations discovery. Predictive modeling can help predicting outcomes and forecasting business  trends. Forensic analysis focuses on detecting deviations and analyzing links between trends and patterns.

Literature Review

Despite the short history of data mining, several authors researched the technologies surrounding the business process, as well as its benefits of using knowledge to improve business performance. The below literature review will examine some of the findings revealed by researchers in order to create an effective, low-cost, and efficient data mining framework for companies that would like to benefit from information freely available to them.

Bal, Bal & Demirhan (2011) talk about the topic that is closely related to the current research: the connection between knowledge management and competitive advantage. The author confirm that today’s business environment – due to the impacts of globalization – is more competitive than it has ever been, and knowledge can help companies create strategies based on knowledge obtained through data mining to increase their competitiveness. As the authors conclude: “knowledge is central to strategy formulation and implementation” (Bal, Bal & Demirhan. 2011, p. 4). Further, the authors state that – based on related contemporary theories – “the most valuable assets of the 21st century enterprise are its knowledge and knowledge workers” (p. 4).

Abdellatif,  Elsoud & Ali (2011) created a clear distinction between data mining and other methods of analyzing and processing data (online analytical processing). The main differences the authors found are based on the goals of the analysis, and the questions business professionals are able to get answer to. Comparing questions related to customer discovery through data analysis, the authors found some significant differences. While OLAP (Online Analytical Processing) can answer to the simple question: “What was the response rate to companies’ mailing”, data mining answers a more specific query: “What is the profile of people who are likely to respond to future mailings. (Abdellatif,  Elsoud & Ali, 2011, p. 165)

The above distinction has a great significance to the current research. It confirms the initial assumption that data mining can be used by managers to make strategic decisions. While knowing the response rate to the mailing might be an essential information for a company that spent a lot of money on direct mail, it does not help creating a strategy for the future that can improve results.

The superiority of data mining over OLAP in process improvement and operations management is clearly identified by the authors. Reviewing different applications of data mining, the article found that data mining can support strategic decision making in every field of business. Rupnik,  & Jaklic (2009) state that business data can be implemented into process change design projects in order to improve the overall effectiveness of the system.

Kumar, Tyagi & Tyagi (2014) lists some of the main business goals that can be reached with the support of data mining technologies. These are: data processing, application rule learning, prediction, regression, classification, link analysis and association discovery, clustering, exploratory data analysis, model visualization, summarization, and dependency modeling. Translated to real life business situations, the above methods can analyze business and customer segments separately.

Applications

Pal (2011) lists several applications of data mining, based on the effectiveness of the tool in the particular field. The below list of applications – while it is not comprehensive – will be providing a clear indication of the benefits of data mining in various business fields.

• fraud and non-compliance anomaly detection
• intrusion detection
• lie detection
• market basket analysis
• aid to marketing and retailing
• customer segmentation and targeted marketing
• detecting “beer and baby diapers” phenomena
• credit risk scoring and analysis
• health care and prevention
• corporate surveillance
• scientific research

The above list of data mining application is confirming that knowledge can be used as an aid for strategic planning in many fields of business and government operation. However, as the current study only focuses on business applications, the authors will be closely studying the benefits related to marketing, customer analysis, sales prediction, trend and risk analysis.

Customer relationship management is one of the most commonly examined application of data mining. Increasing customer value can not only reduce the cost of marketing, as it is easier and cheaper to keep existing customers than gaining new ones, but can also increase a company’s operating profit. Rygielski, Wang & Yen (2002, p. 493) list three different ways of increasing customer value during the life cycle. There are three methods listed by the authors that can increase the value of individual customers: making them use the product more, selling them products with higher profit margin, and keeping them longer. All of the above strategies can be supported by data mining technologies. As the authors summarize: “Data mining can predict

the profitability of prospects as they become active customers, how long they will

be active customers, and how likely they are to leave” (Rygielski, Wang & Yen, 2002, p. 494).

Related Research

Singh (2012) analyzed managers’ perception of the applicability, value, and benefits of data mining. According to the author (Singh, 2012, p. 1), 61 percent of managers were aware of problems caused by information overload. Further, 81 percent of them believed that – as new technologies continue to evolve – the problems caused by information overload will become worse. Interestingly, however, more than 50 percent of managers ignored data obtained, as they struggled with information overload. As it has been previously confirmed, data gained using data mining techniques can help management make the right important strategic decisions. Therefore, it seems like the majority of decision-makers are passing on an opportunity to improve the business, as they do not have relevant tools that can help them analyze large amount of data. The implication of this trend in business is evident: decisions not supported by confirmed trends and patterns will provide a lower strategic value than those gained through data mining. Consequently, the business’ ability to create a competitive advantage will be reduced.

Rygielski, Wang & Yen (2002, p. 496) created a case study that revealed the detailed process of implementing data mining solutions to improve business processes. In the case example, a large retailer suffered profit loss due to the lack of reliable product demand forecasts. By creating a business intelligence system that allowed the company to group retail stores by location and identify historical sales patterns, taking into consideration the location’s demographic features, the management was able to overcome forecasting issues. As a result, they were able to improve their supply chain management, and could predict demand. This resulted in matching inventory closer with market demand. The company reduced operation expenses and surplus, and increased its revenues by 11.6 percent within a year (p. 497).

Janakiraman & Umamaheswari (2014) created a survey to analyze customer relationship management utilization of data mining. Reviewing related literature, the authors found that the most commonly used methods of data mining by customer relationship managers are clustering, classification, and prediction. Reviewing recent related studies, the authors have found the following limitations of  the three techniques:

• only effective to handle large sets of data
• robust to noise
• methods assume the independence of different variables

Sharma, Kaur, & Manju (2013) created a life cycle model for data mining. According to the authors, there are two main phases of data mining: business research and understanding. Business research is the first step, followed by deployment, evaluation, modeling, data preparation, and data understanding. There are several connections between the phases, and the success of data understanding depends on the quality of implementation within the previous phases. Likewise, if data is not prepared in a way that it can create answers to business questions, the business understanding phase cannot be completed.

Benefits Identified

The article created for business managers and operations managers by MathEpic (2014) list several case studies related to the benefits of data mining in business. The examples are from variety of industries and business sectors. Some of the interesting cases and the identified benefits of data mining using will be listed below.

Boehringer Ingelheim, a large pharmaceutical company managed to reduce drug discovery time as a result of data mining software implementation. HSBC, a global financial organization has successfully identified key clients, reduced marketing cost by 30 percent, and increased sales by 50 percent. The authors of the study highlight one important connection between customer relationship management and data mining: “CRM practices through data mining allow the bank to use 20% of its resources to generate 80% of it’s profit” (MathEpic, 2014, p. 2). Even Facebook, the social media giant uses an external data mining company to measure the conversion rate of their advertisements and make strategic decisions about re-targeting campaigns.

Similarly, Nejad, Nejad & Karami (2012, p. 5012) state that “data mining tools can answer business questions which were time-consuming to track in the past”. The authors highlight that data mining can not only determine the value of customers, but also buying patterns and trends in customer behavior. Using the well-known 20-80 approach to business, companies can focus on the 20 percent of customer relationship management activities that can bring in 80 percent of the results. This will not only increase customer engagement (something that many online and offline marketers talk about in the 21^st Century), but can also be a valuable tool for reputation management.  The authors (Nejad, Nejad & Karami, 2012, p. 5013)  – based on the needs of customer relationship management’s information needs- created a list of advantages that data mining can deliver companies:

• answering customer related questions and queries quickly
• elimination of duplicate data and cleansing records
• advanced customer profitability analysis capabilities
• data integration for better reporting capabilities
• calculating present and future values of customers
• fast response to business environment changes and challenges
• increased loyalty and customer satisfaction
• attracting new customers through advanced market segmentation

Ranjan (2006) lists the benefits of business intelligence systems and data mining as follows: eliminating guesswork when creating marketing and market strategies, responding to challenges of business environments faster, simplifying decision-making, and identify problems in supply chains, quality assurance systems, and other important risk areas, such as fraud and money laundering.

Piller & Hagedorn (2014) talk about the capabilities of in-memory data management. The authors find that potential benefits of data analyzed by business intelligence systems are present in operational reporting, adaptive planning, and data from consumer devices. The less flexible application of data mining was found to be operational reporting.

Joseph (2013) states that several business problems can be addressed through data mining that were previously hard to solve, including marketing and customer relationship management issues, fraud detection, and network management efficiency problems. Focusing on the telecommunication industry, the author concludes that companies can now detect and predict network faults, measure service availability, and tailor their services to customers’ telecommunication behavior patterns.

Further, Jackson (2013, p. 272)   lists some of the common objectives of businesses when implementing data mining, such as: lowering product development costs, improving statistical control methods in manufacturing, eliminating marketing for non-responding customers and saving money on direct mail, or identifying mass communication opportunities. The same case is analyzed by  Rupnik,  & Jaklic (2009), who found that the “use of Data mining will enable new ways for customer segmentation and discovering customer groups for marketing campaigns” (p. 384).

Challenges Identified

Apte (2011, p. 18) uses the term: “big data challenge”, to describe a trend that managers simply call “information overload”. The main aspects of the challenge are: data growth, format variability, structure, and cost of retrieval. The research also states that moving large amount of data safely at the moment is expensive. Further, the time sensitivity of data is increasing, in particular when conducting online research and analysis. Creating and updating various algorithms to gain insights of markets is also an expensive process.

Kumar, Tyagi & Tyagi (2014) find the majority of challenges technological. They are related to how clean the data gained is, how easy to access it, and how the company can obtain skills for employees to interpret the data gained from knowledge information systems. Further, data ownership, distribution, and privacy issues can create a challenge for both companies and those providing information for corporate customers. However, it is also important to note that the processes and technology surrounding data mining are complex, hard to understand, and companies are likely to need to invest into training managers and analysts to make the most out of the capabilities of systems available.

Further, as the benefit of data mining for companies is not widely documented, several decision makers believe that projects involving knowledge discovery systems have a low return on investment (ROI) ratio.

Sharma, Kaur, & Manju (2013) state that there are several challenges related to the technologies currently deployed to support data mining. Human interaction is one of the main challenges. The majority of systems are not designed for average users, and those trying to utilize systems and software to increase business knowledge capabilities will often need to be technical experts. This is a great challenge, and it is important that companies developing data mining software do so with the end user in mind. Over-fitting can also create a major distraction and increase customers’ knowledge discovery budgets. This problem occurs when software developers create an information discovery system that closely matches the current requirements of the company. While this solution can be ideal for short term, its inflexibility can cause problems when data formats and business demands for information change. The article also lists privacy concerns as one of the main obstacles ahead of the future development of data mining technologies. As privacy breaches and customer data protection can have a great impact on the company’s reputation, the area will be covered later in the paper, in order to reveal the most effective solutions. Data integrity, interpretation, visualization of results can also create great challenges for analysts and managers alike. Further, the article highlights the fact that applying algorithms on large data sets is more problematic than dealing with simple internal company records. The complexity of algorithms and the appearance of multimedia data has also created new challenges for the industry. The authors (Sharma, Kaur, & Manju, 2013, p.698), however, state that data mining is only useful for companies when its application can create benefits for the business. As data mining is an emerging technology, without a proven “track record” of effectiveness, companies are less likely to believe that it can deliver the results expected. That lack of trust, combined with the high cost of data mining can create an obstacle for future industry development.

Khabaza (2005) states that the main challenge for companies is getting “lost” under the mountains of data. Indeed, it has already been noted that the majority of managers are today suffering from information overload, and find it difficult to distinguish between relevant and irrelevant data: datasets that can help improving the business when implemented in strategic decision making, and data that is simply collected but cannot create a value for the organization. Companies also need to ensure that data mining processes are clearly organized, and information is easy to retrieve. The author  (Khabaza, 2005) also mentions one particular problem that has not been mentioned by other researchers, and is extremely relevant to the current review: “insufficient business knowledge” (p. 5). The solution suggested by the author is to involve an end user and a business manager of the area from the planning stage in the data mining design and implementation, as – in order to provide the right answers – the system needs to be based on queries that can improve processes and company performance. Finally, one technological challenge is mentioned by the author: data mining tools’ incapability with several types and formats of data collected from different sources.

Trends Identified

Social media data mining is considered to be one of the main trends of the 21^st Century. As IBM’s research (2011, p. 30) states: “8 out of 10 bloggers post product or brand review”. This indicates that extracting data from blogs and social media is not only useful for companies to identify customer behavior and market trends, but can also support reputation management initiatives.

Nejad, Nejad & Karami (2012) highlight the importance of customer relationship management in modern business. The authors state that data mining is an important and effective tool for customer relationship managers. One important trend that the article highlights is that “rapid internet growing and its related technologies have changed interaction and communication ways between companies and their customers” (p. 5011). This, in turn, can provide new challenges and opportunities for companies, and assessing data about customer behavior, buying patterns, and segmenting the customer base can be important for marketing, as well as product development.

Sharma, Kaur, & Manju (2013) identified some important growth factors in the industry and technology of data mining. The growth of the industry is accelerated by the growth in companies’ data collection activities, development of data warehousing technologies, web navigation intranets, the impact of globalization on companies’ need to become more competitive, and finally the rapid growth of storage and computing capabilities. While the list above is related to data mining and its technologies, it is missing one important element, which will later be discussed in this paper: online storage and collaboration technology trends, such as cloud computing. Likewise, Al-Azmi (2013, p. 3) confirms the rapid development of internet-based data collection as follows: “developments will lead to the more intelligent agents that search the WWW for not only keywords but also site visitors’ patterns”.

Technologies

Singh (2012, p. 3) categorizes data mining techniques as follows: Classification, Regression, Clustering, Summarization,  and Association Rules.

Classification is a method used to determine trends. It can be used in the financial industry, for example, to retrieve patterns from a large amount of data. The classification algorithm is the most commonly used by businesses, and is extremely popular among financial firms.

The regression technique is used to determine connection between different variables. While it is an effective prediction model, often there is a need to use more than two variables to identify complex relationships between data sets. As Singh (2012) confirms, making predictions on only two variables is difficult. The author uses the example of trying to predict sales price trends. It is evident that the price will depend on competitor companies’ marketing, customer behavior, company reputation, and other variables as well.

The clustering algorithm is an effective tool to create categories for large amount of data. An example would be separating sales analysis for international and domestic clients, so the company can create an individual strategy for each segment of the market.

The summarization technique is useful when companies need to identify basic trends, such as “tabulating the mean and standard deviations” (Singh, 2012, p. 3).

The Association rule is applied when companies are trying to identify frequent items within large datasets. It can be, for example, used to find cases when products were returned, and analyze the data based on how frequently each product was returned by customers. Companies can determine the “problematic” products and either improve them or stop selling them.

Jackson (2002) clearly identifies the actors needed for successful data mining implementation. While software and algorithm development is crucial to gain data and analyze it, companies should not forget about the human element of business: the knowledge gained needs to be implemented into various strategies designed to create competitive advantages. A project leader is needed to plan and schedule the process. A data mining client will be responsible for requesting answers to different queries and utilizing them. A data mining analyst will look at the results from the business’ perspective and align corporate strategies with trends identified. The role of data mining engineers is to evaluate the data collection and analysis model, while the IT analyst provides hardware, software, security software, and creates an infrastructure for data mining.

Foster (1997) uses a simple, three-component visualization of explaining the process of data mining. The author states that the three elements: preparation, discovery, and analysis are overlapping, and dependent on each other. Companies need to prepare data first, so it can be used for statistical and analytical purposes. Next, it is important to select the related technology in order to match business requirements, constraints, allocated costs, and training with the purpose of the project. Information discovery can, and should be automated, according to Foster (1997), as this reduces the human resource cost implication of data mining. There is also a need for reapplication and redeployment of data, in order to validate results. Finally, the knowledge discovery data set is analyzed, assessed, and decisions are made based on the finding.

Ethical Considerations

Most of the ethical considerations are related to customer privacy and the fair usage of information collected online and offline. Yang and Wu (2005) states that apart from compliance issues and fines based on the Data Security Bill of the United States (2005), companies can suffer from reputation loss if they do not use data fairly.

Seltzer (2005) highlights three important areas of business ethics related to data mining and statistical analytics: suitability and validity, privacy and confidentiality, and usage based on the aims of the project. As the author (p. 1442) concludes: “The fact that a procedure is automated does not ensure its correctness or appropriateness” and companies employing statistical research need to consider “the social value of their work and the consequences of how well or poorly it is

performed”, and they have to “avoid excessive risk to research subjects”.

Privacy Issues

Brankovic & Estivill-Castro (1999) identify the main privacy issues of knowledge discovery and data mining as secondary use of personal information, misinformation, and granulated access to personal data. The authors highlight the importance of guarding personal data from knowledge researchers, in order to comply with regulations and privacy policies. While the concern is valid, it is important to note that today’s knowledge discovery and data mining software and tools have built in applications that make only non-personal data available for analysis. Data mining’s main purpose is not to “get to know” individual customers”, but to identify patterns and trends of a large set of data.

IBM’s research (2011) highlights the new and emerging benefits of data mining. One interesting trend identified by the authors is that social media is rapidly changing customer behavior, therefore, companies should utilize information collected through these sources to spot buyer trends, market changes, and respond to them in a timely manner.

Quiu, Li & Wu (2007) talk about one particular issue related to obtaining data through information data discovery and mining services. One of the impacts of globalization is that – in order to reduce operational expenses – many companies outsource business processes, and in many cases the whole IT department. Protecting data through internal privacy regulation is challenging enough, however, when dealing with an offshore company, it becomes even more challenging.

Findings

The above review of the related literature has revealed that the industry of data mining is rapidly evolving, and involving. While technologies are available for all industries to support the decision making process and create competitive corporate strategies, there are also several challenges ahead of the development. The benefits of data mining can create in customer relationship management have been found extremely strong, and several case studies have been reviewed that confirm that companies can create a competitive advantage, in the sales, financial, and even pharmaceutical industries. The research has also revealed that the most benefits can be achieved in marketing and customer relationship management. The below analysis will summarize the main findings of the reviewed studies.

Data mining today has several beneficial capabilities

Data mining’s main benefits for business have been identified as customer research, market research, process improvement, sales strategy development, and error/policy violation detection. Several case studies found that banks have managed to identify financial anomalies and identify fraudulent activities. Companies in the telecommunication industry could better understand customer behavior, network availability, and service level. Retail companies could create a competitive advantage by creating closer prediction for market demand, and could reduce operational costs significantly. Pricing strategies can be supported by data mining. With the development of online platforms, companies can research market trends and identify challenges, as well as opportunities.

Companies can utilize data mining to improve business processes

It has also been found that companies can not only have primary benefits of data mining. The case study of the large retailer has showed that – while inventory and supply chain management improved as a result of using data mining for demand prediction -, the company also managed to realize higher profits, but also increased customer satisfaction, improved its relationship with retailers. Therefore, data mining does not only help spotting and eliminating problems within its operation, but can also benefit from longer term improvements and increase their reputation. As business processes are highly dependent on each other, improving one area can have a positive impact on other aspects. Simply put: improving the targeting of marketing based on data mining’s results about customer and market segments will reduce the costs associated with advertising. The company’s profitability increases, and the company is able to reduce prices, realize more profits for investors, and increase customer satisfaction. Likewise, using data mining to identify financial anomalies and fraud will not only improve the company’s compliance, but can prevent financial loss. The relationship between compliance, shareholder relations, and company reputation does not need to be confirmed; it is clearly visible for all managers.

Data mining is only beneficial if it provides adequate and relevant answers to the business’ questions

Interpretation and analysis of the data has been found as the most crucial part of data mining, and if companies fail to implement the findings of the process into their corporate strategy, they cannot benefit from a higher performance and create a competitive advantage. This creates a problem for data mining implementation: the management has to ask the right questions to find the answers that can help them make informed strategic decisions that will improve the business. That, combined with the technological challenges of retrieving and analyzing data can make the implementation even more problematic.

The most promising areas of data mining implementation is marketing and customer relationship management

Several authors (Nejad Nejad,& Karami, 2012;  Rygielski, Wang & Yen) have examined the capabilities of data mining in relation with customer relationship management. It has been found that this area of implementation is the fastest growing, as online applications have developed rapidly in the past few years, and companies started to realize that data from websites, social media and online shopping applications can help them understand both existing and potential customers. Indeed, the development of internet applications has made customer behavior research, profiling, market segmentation, and even market research in general easier, and most cost-effective. While several companies, such as Facebook use data mining to determine the factors that influence customer behavior, buying patterns, and even online advertising conversion rates, it is important to note that this type of utilization imposes the greatest privacy challenge on companies. As online marketing has evolved to a level that email communications are mostly based on simple double opt-in sign ups, the issue is more complicated in accessing internet usage patterns of customers and prospects. It is likely that new regulations will be created worldwide to protect customers from having their usage patterns or demographic information exposed to companies. Sharing data for customer research also has some potential compliance risks attached, as authors have noted (NASCIO, 2012), and these need to be addressed by companies. Further, outsourcing customer research (Qiu, Li & Wu, 2007) calls for special attention to protect customer personal information.

Data mining is superior to OLAP systems

Data mining has several capabilities that OLAP does not. While OLAP can answer simple questions, often delivered by statistical research, data mining can support strategic decision making in every area of business (Abdellatif,  Elsoud & Ali, 2011). In this sense, it is fair to state that while OLAP answers to the general questions of “what is happening in the business at the moment”, data mining can answer specific, forward-looking ones, such as: “which customers are most likely to respond to online advertising”? Once the answer is provided, strategic decisions can be made to target the specific segment of customers with online advertising. Companies can greatly reduce their advertising cost, avoid “shooting in the dark”, and improve both conversion rates on their online ads and the ROI rates of individual campaigns.

The main obstacles of further development are related to cost, complexity of systems, and the lack of reputation of data mining as an effective business process development tool

The above literature review has revealed that – while many case studies exist regarding the effectiveness of data mining – the majority of managers are afraid of implementing the process because it has a high costs, has a low (perceived) return on investment ratio, and the implementation of the new technologies is problematic. Further, authors have noted that training is required for end-users, and managers, as well as analysts need to actively participate in the development of the business intelligence system. Finally, the complexity of data mining systems, the lack of data compatibility create further challenges for companies that are planning to implement the system. As it has been revealed, the process of creating a business intelligence system based on data mining that can support managers in strategic decision making is lengthy, and all steps are equally important, dependent on each other. If data quality is not acceptable, the system will not provide the right answers to business queries. Likewise, if companies are able to retrieve large amount of data, but cannot distinguish between relevant and irrelevant information, it is likely that a data mining project will cause higher level of “information overload”, and the implementation will be labeled as “useless for the business” and “ineffective”. Without clearly determining business goals in the beginning of the data mining project, the system will not be suitable for providing solutions for the company’s issues.

Discussion

Data mining is often labeled as the technology for the future. However, it is already supporting many organizations worldwide that would like to improve their processes, corporate strategies. Building competitive advantage should be the main purpose of data mining, and many managers find it hard to align business goals with strategies. Data mining has every capability to support companies in this process, while it also has some challenges. The technique can create competitive advantage for businesses not only in one department, but throughout the entire company. Businesses are today viewed as systems, and if we assume that the performance of one department is closely related to the other’s, it is evident that data mining should be aligned with not only the goals of the particular project, but also the corporate strategy.

Globalization has increased the level of competition across all industries. Companies focusing on reducing the cost of research and development in order to increase product profit margin and offer more competitive prices for customers can utilize the new technology.

Many companies focus on customer relationship management, as it can increase profits at a lower cost than obtaining new customers. The importance of understanding prospects has never been more important and easier than now. Data mining offers business intelligence that can offer an insight into the market, customer behavior, and decisions. Still, it seems like the tool is under-utilized by many firms, as it is considered a costly and complicated solution.

Technological development of recent years is substantial, but does not answer all challenges yet. The need for human input previously mentioned does not only increase training and development costs associated with business intelligence, but also human resources expenses. The ability to sort, analyze, and screen data requires skills that not all managers have. Likewise, not all data mining systems are compatible with different formats of data, therefore, their capabilities are limited. Without expert implementation, data mining cannot create a competitive advantage. Companies that focus on simply sales figures and expenses (old school approach) will certainly not see the value of data created and delivered by business intelligence. Data mining requires a systematic and strategic approach to running a business, and this is one of the reasons why it is mostly utilized by large, multinational firms. The other, more obvious reason has already been mentioned is the cost of implementation.

Conclusion and Recommendations

Data mining has the capability to help companies create competitive advantages in various areas of business. However, the cost of implementation and the need for skills, training, and strategic thinking holds back many firms from starting their own business intelligence utilization project. There is a need for further research that would not only vaguely list the (assumed or proven) benefits of data mining for businesses, but provides project managers with examples where the return on investment ratio of the project was high, and the entire company’s performance improved due to the answers provided by data miners.

Several challenges are still ahead of data mining: both companies offering the infrastructure and design, and those looking to take advantage of its capabilities. Future research analyzing the effectiveness, user-friendliness, and – most importantly – flexibility of data mining processes is needed. In today’s globalized world, today’s business problems and questions might not be relevant to the market environment of tomorrow. Companies should focus on making systems user-friendly, easy to customize, utilize, and change in order to make them more competitive over other business intelligence tools. Further, all systems developed should take into consideration the multiple formats of data that can be retrieved from company records, online applications, and social media. This statement is particularly relevant to the marketing and customer relationship management application of data mining. Finally, privacy and security risks need to be addressed by systems, and they need to be aligned with individual business goals. The various utilization methods, formats, approaches, and processes make data mining a challenging task today, and it is up to developers to make systems more appealing for managers and executives, in order to increase the popularity of the business intelligence tool.

References

Abdellatif, T., Elsoud, M.& Ali, H. (2011) Comparing online analytical processing and data mining tasks in enterprise resource planning systems. IJCSI International Journal of Computer Science Issues, Vol. 8, Issue 6, No 2, November 2011 pp. 161-174

Al-Azmi, A. (2013) Data, text, and web mining for business intelligence: A survey. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.3, No.2, March 2013

Apte, C. (2011) Data Mining has helped us to provide competitive advantage in business. Retrieved from https://www.siam.org/meetings/sdm11/apte.pdf

Bal, M., Bal, Y. & Demirhan, A. (2011) Creating competitive advantage by using data mining technique as an innovative method for decision making process in business. Annual Conference on Innovations in Business & Management London, UK, 2011

Brankovic, V. & Estivill-Castro (1999) Privacy issues in knowledge discovery and data mining. in ‘Australian Institute of Computer Ethics Conference (AICEC99)’, Melbourne, Australia, pp. 89-99.

Foster, K. (1997)Digging for gold: Business usage for data mining. CoreTech Consulting Group, Inc.

Jackson, J. (2002) Data mining: A conceptual overview. Communications of the Association for Information Systems Volume 8, 2002 pp. 267-296

Janakiraman, S. & Umamaheswari, K. (2014) A Survey on data mining techniques for customer relationship management. International Journal of Engineering, Business and Enterprise Applications, 7(1), December 2013- February 2014, pp. 55-61

Joseph, M. (2013) Data Mining and business intelligence applications in telecommunication industry. International Journal of Engineering and Advanced Technology (IJEAT) ISSN: 2249 – 8958, Volume-2, Issue-3, February 2013

Khabaza, T. (2005) Hard hats for data miners: Myths and pitfalls of data mining. SPSS. Retrieved from http://www.spss.ch/upload/1113911601_data_mining_khabaza%20(3).pdf

Kumar, A., Tyagi, A. & Tyagi, S. (2014) Data mining: Various issues and challenges for future. A Short discussion on Data Mining issues for future work. International Journal of Emerging Technology and Advanced Engineering. Volume 4, Special Issue 1, February 2014

MathEpic (2014) Data mining. Benefits for business. Retrieved from http://mathepic.com/Data%20Mining%20benefits%20for%20business.pdf

NASCIO (2004) Think before you dig: Privacy implications of data mining & aggregation. Retrieved from http://www.nascio.org/publications/documents/nascio-datamining.pdf

Nejad, M., Nejad, E. & Karami, A. (2012) Using data mining techniques to increase efficiency of customer relationship management process. Research Journal of Applied Sciences, Engineering and Technology 4(23): pp. 5010-5015, 2012

Oszlak, C. & Ziemba, E. (2007) Approach to building and implementing business intelligence systems. Interdisciplinary Journal of Information, Knowledge, and Management. Volume 2, 2007

Pal, J. (2011) Usefulness and applications of data mining in extracting information from different perspectives. Annals of Library and Information Studies. Vol. 58. March 2011. pp. 7-16

Piller, G. & Hagedorn, J. (2014) Business benefits and application capabilities enabled by in-memory data management. Lecture Notes in Informatics. Retrieved from http://subs.emis.de/LNI/Proceedings/Proceedings193/45.pdf

Qiu, L., Li, Y., Wu, X. (2007) Protecting business intelligence and customer privacy while outsourcing data mining tasks. Springer.

Ranjan, J. (2005) Business intelligence: Concepts, components, techniques and benefits. Journal of Theoretical and Applied Information Technology. Vol. 9 No. 1. pp. 60-70.

Rupnik, R. & Jaklic, J. (2009) The Deployment of data mining into operational business processes. In: Data Mining and Knowledge Discovery in Real Life Applications, Eds. Julio Ponce and Adem Karahoca. ISBN 978-3-902613-53-0, pp. 438, February 2009, I-Tech, Vienna, Austria

Rygielski, C., Wang, J. & Yen, D. (2002) Data mining techniques for customer relationship management. Technology in Society 24 (2002) pp. 483–502

Seltzer, W. (2005), The promise and pitfalls of data mining: ethical issues. In: Statistics and Counterterrorism: The Role of Law, Policy and Ethics” 2005

Proceedings of the American Statistical Association, Section on Risk Analysis [CD-ROM], Alexandria, VA: American Statistical Association.

Sharma, B., Kaur, D. & Manju, M. (2013) A review on data mining: its challenges, issues and    applications. International Journal of Current Engineering and Technology. Vol.3, No.2 (June 2013) pp. 695-700.

Singh, H. (2012) Implementation benefit to business intelligence using data mining techniques. International Journal of Computing & Business Research ISSN (Online): 2229-6166

Yang, Q, & Wu, X. (2006) 10 Challenging Problems in Data Mining Research. International Journal of Information Technology & Decision Making Vol. 5, No. 4 (2006) 597–604