Design and Development of a Knowledge Discovery System in Inventory Management C. A. Mitrea, C. K. M. Lee Division of Systems and Engineering Management, School of Mechanical and Aerospace Engineering Nanyang Technological University, Singapore ckmlee@ntu.edu.sg mitr0002@ntu.edu.sg Abstract - To manage the inventory efficiently, it is necessary to have accurate forecasting. To extract and deploy the knowledge associated with forecasting attracts the attention of both academic and practitioners. Knowledge is regarded as a valuable asset for enterprises and it can be manipulated through intelligence techniques like Artificial Neural Networks (ANN). ANN has the special ability to learn facts about one knowledge domain by inputting data obtained from observations. This study focuses on exploring how ANN learns and analyzes different types of ANN and ANN architectures used in the demand forecasting. The feasibility of the proposed approach is demonstrated with numeric data. The significance of this study is to adopt ANN as a knowledge discovery system thereby enhancing the inventory management. the demand forecasting Keywords – Forecasting, Inventory Management, Knowledge Discovery Process, Neural Networks, Rule Extraction, Weights issue to 1. INTRODUCTION is efficiency Inventory management intelligent systems. This has been already affected considerably by the forecasting accuracy which can be enhanced if knowledge about its variables is available. A form of knowledge which is easy to understand by humans is rules. Whether the system is efficient depends on the quality of rules in the system. However, knowledge acquisition is not easy. In the inventory management field, some knowledge cannot be described easily by knowledge engineers, some cannot be understandable easily by managers, and some cannot be identified easily by the bottleneck of knowledge management. Considered as a tool of machine learning, an artificial neural network has the capability to self learning hence acquiring rules by itself through the process of inductive machine learning. However, rules obtained by neural networks are hidden in the architecture and weights of networks, and are not easy to understand by human being or to be applied to improve inventory management [1]. Two crucial criteria for NN learning are based on comprehensibility of learned models and predictive accuracy. While NN attains high predictive accuracy, the incomprehensibility of its predictive behavior prevents its application in solving in inventory management problems. 2. LITERATURE REVIEW For this reason, this research is exploring the techniques and methods to extract rules, embedded in the weights of a neural network which was trained to forecast demand and lead time in inventory management. This section provides a comprehensive study of three topics which are inventory management, knowledge discovery process and ANN. Inventory Management requires detail planning, controlling and monitoring the flow of and storage of goods and service. To achieve satisfactory level of service, implicit and tacit knowledge is required so as to enhance inventory management. Knowledge Discovery Process the to knowledge are processes of examined. Among (AI) techniques, Artificial Neural Networks is identified in this research as the engine to acquire the knowledge. 2.1 Inventory Management is the art of managing inventory which is defined as a physical resource that a firm/company holds in stock with the intent of selling it or transforming it into a more valuable state [2]. Over the past fifteen years, inventory management has become an important focus of competitive advantage for firms and organizations. transforming data is studied and the Artificial Intelligence the enterprise because According to Sprague [3], inventory management plays a crucial role its performance improvement can enhance overall supply chain activities. Liberman [4] identifies five inventory management performance factors and the challenge faced by the companies (1) Technical factors- Companies lack the availability, reliability and knowledge of efficient technology; (2) Organizational factors- Companies may not have right technical input and financial support; (3) Financial factors- Companies may not have explicit financial mechanisms; - Companies do not have sufficient training and proper management, and (5) Informational factors – Companies lack appropriate information sharing problems. inventory Another management performance conducted by Vries [5] has as grouped followings (3) Organizational categories. Similarly, in the research of Rajeev [6], he identifies factors such as: (1) Safety stock, Purchasing (2) Capacity inventory management (1) Economic, performance (2) Behavioral and information and have (4) Managerial influencing study on utilization factors factors to level, (3) 1449 978-1-4244-4870-8/09/$26.00 ©2009 IEEE

effectiveness, (4) Demand forecasting accuracy, (5) Practices of inventory management, (6) Management attitude, (7) Employees training, (8) Interaction with suppliers, (9) Interaction with customers and, (10) Supplier empowerment. Other researchers narrowed the number of inventory management performance factors to technical aspect, and studied only the influential factors in forecasting. Safety Stock which is the extra units of inventory carried as protection against possible stock-outs [7]. While making the order quotation and calculating production plan, the safety stock must be used to guard against the uncertainty of the manufacturing processes, demand and lead time [8]. Some factors such as lead time, number of customers, customer supply reliability can affect the safety stock [9]. Dongxu [10] identified the factors such as use frequency, quality grade and actual safety stock, while Jia [11] suggested forward factors such as stock-out cost, sales situation. Figure 1 is an integration of all influential factors that determine safety stock. Not all the factors that are influencing safety stock performance have the same relevance weighting. Factors like lead time could have a bigger magnitude on the determination of safety stock. To determine the significance of each factor into a causal NN forecasting model is one of the objectives of this research. satisfaction, delivery reliability, 1.2 Knowledge Discovery Process (KDP) is a non- trivial process of identifying valid, novel, potentially in data [12] useful and understandable knowledge Researchers have proposed a seven – step approach is described in Figure 2 [12]. A brief description of each step is shown as following: 1) Goal identification. Goal identification has the purpose to clearly define what should be accomplished. Goal identification is the most difficult task, as decisions about resource allocations as well as measures of success need to be determined. A clear problem statement is enlisted as set of criteria to measure success and failure. 2) Data selection - Data selection step restricts subsets of data from larger databases and different kinds of data sources. This phase involves sampling techniques, and database queries. 3) Data Preprocessing – data preprocessing represents data coding, enrichment and clearing [13] which involves accounting for noise and dealing with missing information. Usually the majority of data preprocessing takes place before data is permanently stored in a structure such as data warehouse. Figure 2 Knowledge Discovery Process steps 1450 Figure 1 Influential factors of safety stock that can be solved 4) Data transformation- has the purpose to change data into a form, from which useful knowledge can be extracted. Data transformation methods also reduce the dimensionality of data and eliminate statistical properties that are harmful to data mining techniques. 5) Data mining step is made of three tasks: specification, selection of an appropriate method, and problem solving. The data mining task is an abstract description about the problem through classification, clustering, association and prediction. 6) Evaluation and conclusion making is a user-oriented step where the analyst interprets the model; verifies its stability and validity, and evaluates the interestingness of the result. The model can be evaluated quantitatively with respect to data and formalized assumptions. 7)Use of knowledge- This is the final objective of Knowledge Discovery Process which involves user action, such as minimizing Safety Stock but maintaining the same customer service level or even improving it according to the factors that are relevant to inventory management performance. is an information processing certain performance characteristics in common with biological neuron networks. 1.3 Artificial Neural Networks (ANN) that has system Neural Network with hidden layers is universal approximators, which means that, in theory they are capable of learning an arbitrarily accurate approximation to any unknown function, provided that they increase in complexity at a rate approximately proportional to the size of the training data. Neural networks can be applied to time series modeling without assuming a priori function forms of models. A variety of neural network techniques have been proposed, investigated, and successfully applied to time series prediction and causal prediction shown in Figure 3. Proceedings of the 2009 IEEE IEEM

(a) (b) (b) (a) (b) (c) (d) Figure 3 NN used in forecasting Multilayer Feedforward NN (MFFN) (a) [14] is the most common NN used in causal forecasting, the flow of information is from the input layer to the output layer, Recurrent NN (b) [17, 18] is basically a Feedforward NN with a recurrent loop, therefore the output signals are fed back to the input, Time delay NN (c) [15,16] integrates time delay and Nonlinear Autoregressive eXogenous NN (NARX) (d) [19,20] is a combination of all above NN, and is applied successfully in time series forecasting and also causal forecasting. It will show in Chapter 3. lines, The use of NN in forecasting can be described intuitively as follows. Given a certain amount of historical data which can be used to analyze the behavior of a particular system, such data can be used to train a NN to correlate the system with respect to time or other system parameters. Even this seems a simplistic description, experiences shows that NN approach is able to provide a more accurate prediction than expert systems or statistical counterpart [21]. though 3. METHODOLOGY This section proposes a knowledge discovery model shown in Figure 4 and a computational example using a given set of data [22]. Table 1 is the extract of first 9 values from 48 total values. Period (month) Season factor Level Trend Demand Random Demand 1 0.47 18439 524 8000 21234 2 0.68 19015.4 524 13251.16 34345 3 1.17 19644.2 524 24087.26 4 1.67 20325.4 524 36306.13 9643 44321 Figure 4 Knowledge Discovery Model Knowledge Discovery Model used is composed from nine steps. This paper focuses only on first seven steps. Rule extraction step will be analyzed in depth in future work. Decompositional approach is based on an extraction algorithm in which each component of the network is examined. The knowledge extracted at this level is combined afterward. As this method concentrates on the individual components, this is considered an open-box approach. Pedagogical (input/output) approach concentrates on the analysis of the input /output behavior of the network. In essence, the method pertains to of the black box approaches. The purpose of this experiment is to demonstrate that a neural network learns facts about relationships between the input variables and output ones, if the relation exists. Therefore after testing different NN architectures a NARX network is selected, with 17 neurons in the hidden layer and having four variables as inputs: Period, Season factor, Level, Trend and Demand as output (Table 1). In the hypothesis, it is assumed that there is a relation between input variable and output safety stock (S) [22]: equation Winter described t+ , where F nt+ is the forecasted F l demand for period l, L t is the level, T t is the trend, and t+ =(L t +lT t )S l equation by 5 0.47 21059 524 t+ is the seasonal factor at the time period l . According S l 10784.34 to Winter’s model the equations that govern L, T and S 12321 are described below: 6 0.68 21845 524 16458.04 7 1.17 22683.4 524 29850.21 8 1.67 23574.2 524 44881.92 9 0.47 24517.4 524 13296.39 53423 16897 33873 43212 Table 1 Tahoe Salt data 1451 Proceedings of the 2009 IEEE IEEM

+ 1 + 1 + 1 + 1 + 1 + / − = = L t T t S α D ( t β L ( t γ = ( To simplify the model we assume that T and S are constant, and the periodicity of demand is 4. −+ α L ) 1( )( t β −+ T ) 1( t γ −+ S ) ) 1( S + t 1 L ) t L / t + 1 D t ++ pt 1 T t + 1 t ) For example the forecasted demand at period 2 is 1 2 calculated as: F =2 ST+ )2 (L 1 Therefore in training the network, the input layer is fed with 4 variables: L1, T1, l(time period) and S2, while the output is F2. =(18439+2*524)0.68=13251.16 The network uses the default Levenberg-Marquardt algorithm for training. The application randomly divides input vectors and target vectors into three sets: 60% are used for training, 20% are used to validate that the network is generalizing and to stop training before overfitting; the last 20% are used as a completely independent test of network generalization. 4. RESULTS After training the network, the performance can be evaluated using performance function. In Figure 5, it can be observed three lines representing: train set (blue), validation set (green) and test set (red). Mean Squared Error (MSE) is used as performance measurement. The goal of the network in training step is to minimize MSE, while avoiding becoming over-fitted. The training process is stopped when the validation set is with minimum MSE (ideally 0). At that moment, the test set is applied (red color). The feature of the NARX network to learn is tested in 2 situations (a) and (b). In the first case there is a relationship between input and output variables as the output is calculated with Winter’s equations. In the second case (random demand), there is no relation between output demand and input variables. We can observe that the network is not trainable shown in Figure 5. Therefore, it will not learn a pattern between inputs and outputs variables. Testing the trained NARX network and the demand forecasting for period 50-53 is shown in Table 2. The outputs of NARX network are depicted in Table 3 and Figure 6 shows the comparison between actual and forecasted demand with NARX. Period Season Level 50 0.68 51 1.17 52 1.67 53 0.47 105213.4 108305 111449 114645.4 Trend 524 524 524 524 Table 2 Demand forecasting for periods 50-53 Actual demand NARX forecasting 89361.112 157983.93 231623.99 66936.178 101643.6233 198907.639 207617.6869 98749.672 Table 3 Difference between actual and forecasted demand VS Figure 5 Comparison between actual demand (b) (a) Figure 5 Performance analysis Figure 6 Comparison between actual and forecasted demand with NARX 1452 Proceedings of the 2009 IEEE IEEM

5. DISCUSSION In the theoretical example, a simple relation (Winter’s equation) is implemented between input and output data, the NARX network is able to detect the relation demonstrated and it shows that NN has a good learning ability. In the second case, there is no relation between input and output variables. Therefore the network is unable to be trained. The ability of the NN to be train is proportional with the number of the training samples. In our case 48 sample dataset is used. If the number of samples is increased, the learning performance can be further improved. 6. CONCLUSION The paper adopts computational example to show that a neural network is able to detect the hidden knowledge. Apart from detecting knowledge embedded in NN, NN can be used in discovering relations between inputs parameters successfully. However, knowledge extraction and representation in an understandable way for humans needs further investigation. Therefore the further research is to explore how to represent knowledge related to investigate either inventory management and Decompositional or Pedagogical approach is more appropriate in this study. Neural networks models shows promising results in increasing the forecast accuracy, both in time series and causal models. Extracting the knowledge embedded in NN weights could help in understanding the relevance of input parameters and therefore the factors which are determining inventory management efficiency. The process employed with extracting knowledge from trained NN is Knowledge Discovery process. In this paper, the algorithm of knowledge discovery process has been analyzed by studying responsible with collecting, processing and deploying of knowledge. First, and foremost, I would like to express my sincere gratitude and appreciation to my supervisor, Dr. Carman Lee time, unmeasured patience, inspirations and advices throughout this entire process. She was always there pushing me, challenging me and encouraging me in every possible way. ACKNOWLEDGMENT the steps for her to REFERENCES [1] R. Andrews, J. Diederich, A.B Tickle, “Survey and critique of techniques for extracting rules from trained artificial neural networks”, Knowledge-Based Systems In Knowledge- based neural networks, Vol. 8, No. 6. (December 1995), pp. 373-389. [2] D.M Lambert, J.R Stock, ”Strategic Logistics Management”, 3rd edition, Irwin, 1993. [3] L.G Sprague and J.G. Wacker –“Macroeconomic analyses of inventories: Learning from practice” International Journal of Production Economics, Volume 45, Issues 1-3, 1 August 1996, pp. 231-237 of in levels [4] B.M Liberman, S. Helper, “Empirical determinants of inventory high-volume manufacturing”, Production and Operations Management, Vol 8, No. 1, pp.44-55 [5] J. Vries, “The complex relationship between inventory control and organizational settings: theory and practice”, International Economic, Volumes 93-94, 8 January 2005, Pages 273-284 Production Journal [6] N. Rajev “ Inventory management performance in Indian machine tool SMEs: what factors do influence them?” IEEM, 2008. [7] S. Viswanathan, R. Piplani “Coordinating supply chain inventories replenishment epochs”, European Journal of Operational Research, 2001: pp.277- 28 through common [8] Yi-Feng Hung, for Semiconductor Manufacturing”, Electronics Manufacturing Technology Symposium, 1996. “Determining Safety Stock [9] Yu Jinping, Weng Falu, “The Research about the safety stock Based on the BP neural network” Agriculture Network Information, 2004, 11: pp.24-26 [10] Wang Dongxu, Shen Yimin, Wang Zhiqiang. “A Model of Forecasting Safety Stock in ERP Based on BP Neural Network”, Journal of Computer Applications, 2001, pp.53- 55 [11] He Jia, Wang Ziniu, Luo Gang, Zhang Nan, “Research about the Forecasting of Safety Stock Based on Chaotic Neural Network”, 2007 [12] U.M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, “From Data Mining to Knowledge Discovery: an overview”, Advances in Knowledge Discovery and Data Mining, 1996, pp.1-34. [13] P. Adriaans, D. Zantinge, “Data-mining”, Addison Wesley Longman, England, 1996. [14] N. Davey, S.P Hunt, R.J Frank, “Time series prediction and neural networks”, Journal of Intelligent and Robotic Systems, vol. 31, 2001, pp 91-103 [15] E. Wan, “Finite impulse response neural networks with applications in time series prediction”, Stanford University, 1993 [16] A.D Back, E. Wan, “A unifying view of some training algorithms for MLP with FIR filter synapses” IEEE Press, New York, 1995, pp.146-154. [17] Y. Bengio, P. Frasconi ,P. Simard, “The problem of learning long-term dependencies in recurrent networks” Proceedings of the IEEE International Conference on Neural Networks, 1993. [18] J. Conner, R. Douglas , “Recurrent NN and robust time series prediction”, IEEE transactions on NN, 1994, pp.240-254. [19] M.P. Jose, Jr Menezes, A. Guilherme, “A new look at nonlinear time series prediction with NARX recurrent neural network” IEEE 2006 [20] Tsungnan Lin, G.B. Horne, P. Tino and C. L. Giles,“ Learning long term dependencies in NARX recurrent neural networks”, IEEE Transactions on Neural Networks, 1996 [21] H. Bacha, W. Meyer , “ Neural Network architecture for Load Forecasting”, Neural Networks, IJCNN, 1992. [22] Sunil Chopra, Peter Meindl, “Supply Chain Management”, Pearson 2007, pp.200-216 1453 Proceedings of the 2009 IEEE IEEM