Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only. 10379_9789813207516_TP .indd 1
Advances in Process Systems Engineering – Vol. 6
DIFFERENTIAL EVOLUTION IN CHEMICAL ENGINEERING
Developments and Applications
20/4/17 5:54 PM
Advances in Process Systems Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Series Editor: Gade Pandu Rangaiah (National University of Singapore)
Vol. 1: Multi-Objective Optimization: Techniques and Applications in Chemical Engineering edited by Gade Pandu Rangaiah Vol. 2: Stochastic Global Optimization: Techniques and Applications in Chemical Engineering edited by Gade Pandu Rangaiah Vol. 3: Recent Advances in Sustainable Process Design and Optimization edited by D. C. Y. Foo, M. M. El-Halwagi and R. R. Tan Vol. 4: Computation of Mathematical Models for Complex Industrial Processes by Yu-Chu Tian, Tonghua Zhang, Hongmei Yao and Moses O. Tadé Vol. 5: Multi-Objective Optimization: Techniques and Applications in Chemical Engineering (Second Edition) edited by Gade Pandu Rangaiah Vol. 6: Differential Evolution in Chemical Engineering: Developments and Applications edited by Gade Pandu Rangaiah and Shivom Sharma
Herbert Moses - Differential Evolution in Chemical Engineering.indd 1
09-01-17 9:13:37 AM
Advances in Process Systems Engineering – Vol. 6 Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
DIFFERENTIAL EVOLUTION IN CHEMICAL ENGINEERING
Developments and Applications
editors
Gade Pandu Rangaiah National University of Singapore
Shivom Sharma École Polytechnique Fédérale de Lausanne, Switzerland
World Scientific NEW JERSEY
•
LONDON
10379_9789813207516_TP .indd 2
•
SINGAPORE
•
BEIJING
•
SHANGHAI
•
HONG KONG
•
TA I P E I
•
CHENNAI
•
TOKYO
20/4/17 5:54 PM
Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224 USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE
Library of Congress Cataloging-in-Publication Data Names: Rangaiah, Gade Pandu, editor. | Sharma, Shivom, editor. Title: Differential evolution in chemical engineering : developments and applications / edited by Gade Pandu Rangaiah (NUS, Singapore), Shivom Sharma (Ecole Polytechnique Fâedâerale de Lausanne, Switzerland). Description: [Hackensack] New Jersey : World Scientific, [2017] | Series: Advances in process systems engineering ; volume 6 Identifiers: LCCN 2016056698 | ISBN 9789813207516 (hc : alk. paper) Subjects: LCSH: Chemical engineering--Mathematics. | Evolution equations. | Mathematical optimization. Classification: LCC TP184 .D54 2017 | DDC 660--dc23 LC record available at https://lccn.loc.gov/2016056698 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.
Copyright © 2017 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the publisher.
For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.
Desk Editor: Herbert Moses Typeset by Stallion Press Email:
[email protected] Printed in Singapore
Herbert Moses - Differential Evolution in Chemical Engineering.indd 2
09-01-17 9:13:37 AM
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
Preface
Humans inherently optimize their activities in daily life. Optimization can also be seen in natural phenomena and evolution. In chemical engineering, optimization is required in many areas such as model development, design, operation and control. The optimization problem can be for one or more objectives, depending on the requirements of the application. Many optimization problems in chemical engineering are complex in nature due to the presence of non-linearity, large number of decision variables and constraints. Hence, numerical optimization methods are used to solve such optimization problems. Among these, stochastic or metaheuristic methods are simple and useful for solving any optimization problem. An effective optimization method should be able to find the best possible value of the performance criterion by varying decision variables within their bounds whilst also satisfying the constraints in the problem. Differential evolution (DE), proposed by Storn and Price in the year 1995, is a simple and effective method for solving difficult optimization problems. Employing a population of individuals (trial solutions), it iteratively improves them for the given performance criterion through mutation, operation and selection operations. Over the last two decades, DE algorithm has been continually improved in population initialization, mutation and crossover operations, adaptation of parameters, adaptation for multiple objectives and hybridization with other optimization techniques. This book on DE presents the recent developments in DE and its applications in chemical engineering. It consists of 13 chapters, grouped into three
v
page v
April 20, 2017
17:4
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
vi
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
Preface
parts. Introduction chapter (Part I) provides an overview to optimization, DE and chemical engineering applications. Part II (Chapters 2–5) presents a review of DE applications in chemical engineering and DE programs for single and multiple objectives. Finally, Chapters 6–13 (Part III) describe the use of DE for the optimization of chemical engineering applications of importance. The applications covered in Parts II and III, according to the chapter sequence, pertain to chemical reaction engineering, thermodynamics, oil and gas industry, heat exchanger network, separation processes, petrochemicals, fermentation process, metabolic engineering and polymerization reaction engineering. A number of chapters in this book employ DE programs in FORTRAN, MS Excel, R, MATLAB and GAMS, for single and multi-objective optimization. These programs and other relevant materials for many chapters are available on the book’s website (http://www.worldscientific.com/worldscibooks/10.1142/10379). Differential Evolution in Chemical Engineering will be useful for practitioners, researchers and students interested in process optimization and DE. This book offers an overview to process optimization, detailed DE algorithm and use of DE programs for optimizing chemical engineering and related applications. Readers familiar with the basics of optimization and chemical engineering can read any chapter of interest independent of other chapters in this book. In general, we recommend all readers to go through the Introduction chapter. Many chapters in this book can be used as supplementary material in optimization courses for senior undergraduate and postgraduate students, and the associated exercises can be used as assignments or projects. Students can use available programs for solving the exercises in this book or any other optimization problem. Two experts anonymously reviewed each submission/chapter in this book. Then, contributors thoroughly revised, and finally editors read the revised version for consistency. We are grateful to all contributors and reviewers for their cooperation in the timely completion of these activities for a high quality and useful book. Our special thanks to Prof. A. BonillaPetriciolet, for handling reviews of two chapters co-authored by both of us. We acknowledge Lim Swee Cheng, Steven Patt, Herbert Moses and D. Rajesh Babu of World Scientific, for their suggestions and assistance during the preparation of this book.
page vi
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Preface I thank my research students and co-authors for their contributions to this book. I am grateful to all my family members for their deep affection and unwavering support. I am particularly grateful to my wife (Krishna Kumari) for taking care of me and our family. Gade Pandu Rangaiah NUS, Singapore
9in x 6in
b2817-fm
vii
I express my sincere gratitude to my research advisors: Prof. G.P. Rangaiah, Prof. F. Maréchal and Dr. N. Bhandari. I would like to thank my parents, parents-in-law, wife (Dr. Vaishali Gaur), brother and sisters for their everlasting love and support. Shivom Sharma EPFL, Switzerland
page vii
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
b2530 International Strategic Relations and China’s National Security: World at the Crossroads
b2530_FM.indd 6
This page intentionally left blank
01-Sep-16 11:03:06 AM
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
About the Editors
Dr. Gade Pandu Rangaiah has been with the National University of Singapore (NUS) since 1982, in the Department of Chemical & Biomolecular Engineering. He received his Bachelor, Masters and Doctoral degrees, all in chemical engineering, from Andhra University, IIT Kanpur and Monash University, respectively. He worked in Engineers India Limited for two years before his Doctoral study. Dr. Rangaiah’s teaching was recognized with many awards, including the NUS Annual Teaching Excellence Award for four consecutive years. Dr. Rangaiah’s research interests are in modeling, optimization, design and control of chemical and related processes. He supervised 50 graduate theses including 22 doctoral theses. Dr. Rangaiah edited six books, and the second edition of the first book is also published. The three recent books are: “Plant-Wide Control: Recent Developments and Applications” (with V. Kariwala), “Multi-Objective Optimization in Chemical Engineering: Developments and Applications” (with A. Bonilla-Petriciolet), and “Chemical Process Retrofitting Revamping: Techniques and Applications”. He contributed many chapters to these and other books. Dr. Rangaiah published 180+ journal papers and 130+ conference papers in the research area of Process Systems Engineering. For more details on his research and publications, browse http://cheed.nus.edu.sg/stf/chegpr/homefinal.html. ix
page ix
April 20, 2017
17:4
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
x
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
About the Editors
Dr. Shivom Sharma is currently working as a post-doctoral fellow at EPFL (École Polytechnique Fédérale de Lausanne), Switzerland. He received his Bachelor degree from U. P. Technical University, Lucknow. After finishing his Masters degree in Chemical Engineering from I.I.T. Roorkee, he joined the University of Petroleum and Energy Studies, Dehradun and worked there for almost a year. He received his PhD degree (with thesis entitled ‘multi-objective differential evolution: modifications and applications to chemical processes’) from the Department of Chemical & Biomolecular Engineering, National University of Singapore. After completing his PhD under the supervision of Prof. G.P. Rangaiah, he worked as a research fellow in the same research group, for two and half years. Dr. Sharma’s research interests include multiobjective optimization methods, differential evolution, bio-ethanol process, bio-diesel process, water networks and bio-refineries. He has published 12 journal papers, 11 book chapters and presented his work at 13 international/regional conferences. For more details on his research work, browse https://sites.google.com/site/shivomsharmanus/.
page x
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
List of Contributors
Bonilla-Petriciolet A., Department of Chemical & Biochemical Engineering, Instituto Tecnológico de Aguascalientes, Aguascalientes, México, 20256. Chen S.Q., Department of Chemical & Biomolecular Engineering, National University of Singapore, Singapore 117585. Contreras-Zarazúa G., Universidad de Guanajuato, Campus Guanajuato, Departamento de Ingeniería Química, División de Ciencias Naturales y Exactas, Noria Alta s/n, 36050, Guanajuato, Gto., México. Corazza M.L., Department of Chemical Engineering, Universidade Federal do Paraná, Curitiba, Brasil 81531-970. Curteanu S., Gheorghe Asachi Technical University of Iasi, Faculty of Chemical Engineering and Environmental Protection, Department of Chemical Engineering, Iasi, Romania 700050. Dragoi E.N., Gheorghe Asachi Technical University of Iasi, Faculty of Chemical Engineering and Environmental Protection, Department of Chemical Engineering, Iasi, Romania 700050. Errico M., Department of Chemical Engineering Biotechnology and Environmental Technology, Southern Denmark University, Campusvej 55, Odense M 5230, Denmark. Hamedi N., Department of Chemical Engineering, School of Chemical and Petroleum Engineering, Shiraz University, Shiraz, Iran. xi
page xi
April 20, 2017
17:4
xii
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
List of Contributors
Jaime-Leal J.E., Department of Chemical Engineering, Universidad de Guanajuato, Mexico, 36050.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Koop L., Department of Chemical Engineering, Universidade Federal do Paraná, Curitiba, Brasil 81531-970. Maréchal F., Industrial Process and Energy Systems Engineering, École Polytechnique Fédérale de Lausanne, CH-1951 Sion, Switzerland. Quiroz-Ramírez J.J., Universidad de Guanajuato, Campus Guanajuato, Departamento de Ingeniería Química, División de Ciencias Naturales y Exactas, Noria Alta s/n, 36050, Guanajuato, Gto., México. Rahimpour M.R., Department of Chemical Engineering, School of Chemical and Petroleum Engineering, Shiraz University, Shiraz, Iran. Ramírez-Márquez C., Universidad de Guanajuato, Campus Guanajuato, Departamento de Ingeniería Química, División de Ciencias Naturales y Exactas, Noria Alta s/n, 36050, Guanajuato, Gto., México. Rangaiah G.P., Department of Chemical & Biomolecular Engineering, National University of Singapore, Singapore 117585. Rong B.G., Department of Chemical Engineering Biotechnology and Environmental Technology, Southern Denmark University, Campusvej 55, Odense M 5230, Denmark. Sánchez-Ramírez E., Universidad de Guanajuato, Campus Guanajuato, Departamento de Ingeniería Química, División de Ciencias Naturales y Exactas, Noria Alta s/n, 36050, Guanajuato, Gto., México. Segovia-Hernández J.G., Universidad de Guanajuato, Campus Guanajuato, Departamento de Ingeniería Química, División de Ciencias Naturales y Exactas, Noria Alta s/n, 36050, Guanajuato, Gto., México. Sharma S., Industrial Process and Energy Systems Engineering, École Polytechnique Fédérale de Lausanne, CH-1951 Sion, Switzerland. Singh A., Department of Chemical & Biomolecular Engineering, National University of Singapore, Singapore 117585. Sreepathi B.K., Department of Chemical & Biomolecular Engineering, National University of Singapore, Singapore 117585.
page xii
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
List of Contributors
9in x 6in
b2817-fm
xiii
Srinivas M., Department of Chemical & Biomolecular Engineering, National University of Singapore, Singapore 117585.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Torres-Ortega C.E., Department of Chemical Engineering Biotechnology and Environmental Technology, Southern Denmark University, Campusvej 55, Odense M 5230, Denmark. Voll F.A.P., Department of Chemical Engineering, Universidade Federal do Paraná, Curitiba, Brasil 81531-970. Wang F.S., Department of Chemical Engineering, National Chung Cheng University, Chiayi 62102, Taiwan.
page xiii
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
b2530 International Strategic Relations and China’s National Security: World at the Crossroads
b2530_FM.indd 6
This page intentionally left blank
01-Sep-16 11:03:06 AM
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
Contents
Preface
v
About the Editors
ix
List of Contributors
xi
Supplementary Materials
xix
Part I Chapter 1.
1 Introduction
3
Shivom Sharma and Gade Pandu Rangaiah
Part II Chapter 2.
33 Differential Evolution: Method, Developments and Chemical Engineering Applications
35
Shaoqiang Chen, Gade Pandu Rangaiah and Mekapati Srinivas Chapter 3.
Application of Differential Evolution in Chemical Reaction Engineering Mohammad Reza Rahimpour and Nazanin Hamedi xv
70
page xv
April 20, 2017
17:4
xvi Chapter 4.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
Contents Differential Evolution with Tabu List for Global Optimization: Evaluation of Two Versions on Benchmark and Phase Stability Problems
91
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Mekapati Srinivas and Gade Pandu Rangaiah Chapter 5.
Integrated Multi-Objective Differential Evolution and its Application to Amine Absorption Process for Natural Gas Sweetening
128
Shivom Sharma, Gade Pandu Rangaiah and François Maréchal
Part III Chapter 6.
157 Heat Exchanger Network Retrofitting Using Multi-Objective Differential Evolution
159
Bhargava Krishna Sreepathi, Shivom Sharma and Gade Pandu Rangaiah Chapter 7.
Phase Stability and Equilibrium Calculations in Reactive Systems using Differential Evolution and Tabu Search
185
Adrián Bonilla-Petriciolet, Gade Pandu Rangaiah, Juan Gabriel Segovia-Hernández and José Enrique Jaime-Leal Chapter 8.
Integrated Synthesis and Differential Evolution Methodology for Design and Optimization of Distillation Processes Massimiliano Errico, Carlo Edgar Torres-Ortega and Ben-Guang Rong
230
page xvi
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
Contents
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Chapter 9.
Optimization of Intensified Separation Processes using Differential Evolution with Tabu List
xvii
260
Eduardo Sánchez-Ramírez, Juan José Quiroz-Ramírez, César Ramírez-Márquez, Gabriel Contreras-Zarazúa, Juan Gabriel Segovia-Hernández and Adrián Bonilla-Petriciolet Chapter 10. Process Development and Optimization of Bioethanol Recovery and Dehydration by Distillation and Vapor Permeation for Multiple Objectives
289
Ashish Singh and Gade Pandu Rangaiah Chapter 11. Optimal Control of a Fermentation Process for Xylitol Production Using Differential Evolution
321
Laís Koop, Marcos Lúcio Corazza, Fernando Augusto Pedersen Voll and Adrián Bonilla-Petriciolet Chapter 12. Nested Differential Evolution for Mixed-Integer Bi-level Optimization for Genome-Scale Metabolic Networks
352
Feng-Sheng Wang Chapter 13. Applications of Differential Evolution in Polymerization Reaction Engineering
377
Elena-Niculina Dragoi and Silvia Curteanu Index
417
page xvii
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
b2530 International Strategic Relations and China’s National Security: World at the Crossroads
b2530_FM.indd 6
This page intentionally left blank
01-Sep-16 11:03:06 AM
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-fm
Supplementary Materials
To access the supplementary materials for this book, please follow the instructions below. • Go to: http://www.worldscientific.com/r/10379-SUPP (Note: This access URL should be activated ONLY ONCE.) • You will next be prompted to register an account. • Access will be activated upon your account registration. • Access the supplementary materials from: http://www.worldscientific.com/worldscibooks/10.1142/10379#t=suppl. • For subsequent access, simply log in with the same login details in order to access: http://www.worldscientific.com/worldscibooks/10.1142/10379#t=suppl. For enquiries, please email:
[email protected]
xix
page xix
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ebook
This ebook comes with a Supplementary Material, available at the supplementary page http:// www.worldscientific.com/ worldscibooks/ 10.1142/ 10379#t=suppl If you purchase the ebook directly from www.worldscientific.com, simply log in with your login credentials and access the Supplementary page. If you purchase the ebook from Kindle/Apple or any other external sales parties, please use the following access token activation URL: http:// www. worldscientific.com/ r/ 10379-SUPP. You will be prompted to login/register an account. Upon successful login, you will be redirected to the book’s page and click on the ‘Supplementary’ tab to locate the Supplementary Materials.
page 1
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch01
Chapter 1
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Introduction Shivom Sharma1 and Gade Pandu Rangaiah2,* 1
Industrial Process and Energy Systems Engineering École Polytechnique Fédérale de Lausanne, CH-1951 Sion, Switzerland 2 Department of Chemical and Biomolecular Engineering National University of Singapore, 117585 Singapore *
Corresponding author:
[email protected]
1.1 Process Optimization Optimization is an approach to find the best possible solution in the domain of interest while satisfying relevant constraints (restrictions). Optimization problems can be found everywhere, from engineering to economics and from daily life to holiday planning. For example, holiday planning optimization finds the place(s) to visit, when to visit, how to travel and duration of stay (which are all decision variables) to achieve the most happiness (which is the objective function or performance criterion); this may have constraints on budget, travel dates and places to visit as well as other objectives such as safety. Optimization has been fruitfully applied to improve the performance and/or understanding in diverse areas such as science, engineering, business and economics. The goal of optimization is to find the values of decision variables, which will maximize or minimize the value of a given objective function (performance criterion) without violating specified constraints. Mathematically, an optimization problem can be stated as follows. Objective function: Decision variables: Constraints:
Minimize or maximize f1(x) x ≡ x1, x2… xn xL < x < xU 3
(1.1a) (1.1b) (1.1c)
Differential Evolution in Chemical Engineering
4
9in x 6in
S. Sharma and G.P. Rangaiah g(x) < 0 h(x) = 0
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
b2817-ch01
(1.1d) (1.1e)
Here, f1(x) is the given objective function, x is the set of decision variables with xL and xU as the lower and upper bounds, and g and h are the set of inequality and equality constraints, respectively. Many application problems have more than one decision variable and a number of inequality and/or equality constraints. An optimization problem is generally assumed to have only one objective function as in equation (1.1a); such problems belong to single-objective optimization (SOO). Each of these problems will have one or more optimal solutions. Note that optimization refers to both minimization and maximization, and an optimum can be either a minimum or a maximum. A minimization objective can be transformed into a maximization objective by multiplying with –1 or taking reciprocal (with a suitable modification to avoid division by zero). Similarly, a minimization method can easily be modified to a maximization method. Many books describe optimization for minimization, and we follow the same in this chapter. In other words, optimization and optimum are used as synonymous with minimization and minimum, respectively. In the literature, numerous chemical engineering application problems have been optimized for single objective (e.g., see Himmelblau, 1972; Edgar et al., 2001; Ravindran et al., 2006; Rangaiah, 2010; Floudas, 2013). For example, optimization has been successfully applied in the design and operation of chemical and refinery processes, biotechnology, food technology, pharmaceuticals, fuel cells, power plants and bio-fuel production. Capital/equipment cost, operating cost, profit, net present value, energy consumption, efficiency, conversion, yield, selectivity, eco-indicator 99, global warming potential and CO2 emissions are the commonly used objective functions in process optimization problems. 1.2 Classification of Optimization Methods Optimization problems and methods can be classified in various ways using the characteristics summarized in Table 1.1. Some of these are briefly described in the following sub-sections. Many chemical engineering application problems have more than one variable and bounds on variables. Also, they often contain constraints arising from governing equations (such as mass and energy balances, and rate equations) and from process limitations (such as on maximum
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch01
5
Introduction
temperature, pressure and flow rate for safety and due to material of construction). Table 1.1 Characteristics and classification of optimization problems and methods
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Characteristic
Classification
Number of variables: one or more
Single variable or multivariable optimization
Type of variables: real, integer or mixed
Nonlinear, integer or mixed (nonlinear) integer programming Linear or nonlinear programming Unconstrained or constrained optimization
Nature of equations: liner or nonlinear Constraints: no constraints (besides bounds) or with constraints Number of objectives: one or more Derivatives: without or using derivatives Optimum: local or global in the search space Random numbers: without or using random numbers Trial points/solutions: one or more in each iteration
Single-objective or multi-objective optimization Direct or gradient search optimization Local or global optimization Deterministic or stochastic optimization methods Single point (also known as trajectory) or population based methods
1.2.1 Use of derivatives Optimization methods can be classified based on the use of derivate information. If the objective function and constraints are continuous and differentiable, then derivative-based methods such as steepest descent, quasi-Newton and successive quadratic programming (SQP) methods based on gradient vector can be used. These methods are computationally efficient, and give the same solution in different runs if the initial point is the same. Derivative-free methods (e.g., Nelder-Mead or downhill simplex) are used when the objective function or constraints have discontinuities. Both gradient-based and gradient-free methods can be used for solving SOO problems. Some of them are for unconstrained optimization whereas others for problems with constraints. For example, Nelder-Mead, steepest descent and quasi-Newton methods are for problems without constraints, whereas simplex, generalized reduced gradient (GRG) and SQP methods are for constrained optimization. For details on these methods, see Edgar et al. (2001) and Ravindran et al. (2006).
Differential Evolution in Chemical Engineering
6
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
1.2.2 Local and global methods A given optimization problem may have more than one optimum. Fig. 1.1 illustrates this situation for both minima and maxima; in this figure, x-axis represents the search space in one or many decision variables, and the objective function (y-axis) can be for minimization or maximization. There are three local minima, two global minima, four local maxima and one global maximum in Fig. 1.1. By definition, a local minimum is the minimum in its nearby region whereas a global minimum is the lowest minimum over the entire search region (within bounds and satisfying constraints, if any). The objective function in Fig. 1.1 is neither convex nor concave over the entire region, and it is said to be multi-modal.
Fig. 1.1 Local and global optima of an optimization problem
Based on their search capability, optimization methods can be classified into local and global methods. Local search methods generally converge to an optimum in the neighborhood of the initial/starting point. Nelder-Mead, steepest descent, quasi-Newton, GRG and SQP methods are local search methods. These methods require an initial point or solution for starting the search, and converge to a nearby minimum, which can be local or global minimum. On the other hand, global methods search the entire search space, have the capability to escape from the local optimum and to find the global optimum. Multi-start is a simple strategy for searching global optimum using local methods in
Differential Evolution in Chemical Engineering
9in x 6in
Introduction
b2817-ch01
7
conjunction with a number of initial points. Global optimization methods, particularly stochastic methods, may not be successful in every run, and they need more computation time compared to local optimization methods. Randomization is an important component of stochastic search methods. There are many global methods, and they are introduced in the next sub-sections. Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
1.2.3 Deterministic or stochastic methods Optimization methods can also be classified into deterministic and stochastic methods. In deterministic optimization methods, the search for optimum is not random, i.e., it does not depend on random numbers; rather, the search is determined according to the algorithm, optimization problem and initial point. Hence, the new solution found in each iteration does not depend on random numbers. The final/converged solution by a deterministic method depends on the initial point. Examples of deterministic optimization methods are Nelder-Mead (also known as downhill simplex), steepest descent, quasi-Newton, GRG and SQP methods, which are all local methods. They are computationally efficient and can locate the optimum precisely. If the optimization problem is multi-modal as in Fig. 1.1, they are likely to converge to a local minimum near the initial point/solution, thus failing to find the global minimum. Note that deterministic methods require continuous and differentiable objective function and constraints. Stochastic optimization methods, on the other hand, employ random numbers in their search strategies and are more likely to find the global optimum but they generally require more computational time and give less precise optimum compared to the deterministic methods. Almost all of them do not require continuity and differentiability of equations in the optimization problem as well as the initial/starting solution. Hence, stochastic methods can be applied to any type of optimization problems including black-box problems, wherein only the effect of decision variables on the objective and/or constraints is known (and not the underlying mathematical equations and their nature). Since they use random numbers in their search, they may converge to slightly different solutions in different runs. Stochastic optimization methods are based on search using a single point/solution or population of points/solutions, and they are inspired by logic, physical and/or natural phenomena. Simulated annealing (SA), genetic algorithms (GA), differential evolution (DE), particle swarm optimization (PSO) and ant colony optimization (ACO)
Differential Evolution in Chemical Engineering
8
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
are stochastic SOO methods. They are also known as metaheuristics, further discussed in Section 1.3. Although stochastic methods are time consuming, they have become popular due to their applicability to any type of optimization problems, reliability in locating the global optimum, relative simplicity of algorithms and easy adaptability for multi-objective optimization (MOO), discussed in Section 1.4. Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
1.2.4 Number of individuals Another important classification of optimization methods is based on the use of single or multiple individuals at a time. A single point method uses a single individual/solution at a time, which follows a path in the search space as the iterations continue. Hill-climbing and SA are single point optimization methods. Conversely, population-based methods use a number of individuals, and these individuals interact with one another for modifying their respective paths. GA, DE, PSO and ACO are some important population-based methods. Deterministic methods are single point methods. 1.3 Metaheuristics Metaheuristics are designed to solve complex optimization problems for global optimality. The word ‘metaheuristic’ was coined by Fred Glover in 1986. By dictionary meaning, heuristic means to develop rules based on common understanding, and metaheuristic means high level heuristic. Thus, metaheuristics are iterative algorithms incorporating suitable operations and strategies for efficiently finding the global optimum. They use random numbers (stochastic nature), and incorporate exploration and exploitation strategies. A good balance between exploration and exploitation of search space is critical for reaching the global optimum efficiently. Exploration is useful in searching the entire search space and in escaping local optimum, whereas exploitation is required for faster convergence. Metaheuristics can locate the global optimum in reasonable computation time, but they will need infinite time for guaranteed convergence to the global optimum. In the chemical engineering literature, GA, DE, ACO, PSO, SA and tabu search (TS) are commonly used metaheuristics for solving SOO problems. Fig. 1.2 presents a classification of these and some other metaheuristics. Many of them are populationbased, and some population-based metaheuristics are collectively known as evolutionary methods as they are based on nature-inspired concepts.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch01
9
Introduction Metaheuristics
Population-based
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Evolutionary
Single point
Others
Simulated Annealing Tabu Search
Genetic Algorithm
Ant Colony Optimization
Genetic Programming
Particle Swarm Optimization
Evolutionary Strategy
Harmony Search
Local Search
Evolutionary Programming
Bees Algorithm
Guided Local Search
Differential Evolution
Firefly Algorithm Cuckoo Search
Fig. 1.2 Classification of common and recent metaheuristics
Among the evolutionary methods, GA is inspired by the natural evolution of different species (Holland, 1975). Originally, binary strings (or chromosomes) were used to implement GA, where each chromosome in the population represents a trial solution. Each of the chromosomes in the population is randomly initialized within the bounds on decision variables. These chromosomes undergo selection, crossover and mutation operations. Selection operation ensures diversity of mating pool with higher chances of selecting better individuals for crossover and mutation. Crossover operation exchanges information between parent chromosomes, whereas mutation operation adds new random information into the offspring. Subsequently, GA was also implemented using real numbers (Deb, 2001). DE was proposed by Storn and Price (1995) for solving optimization problems over continuous search space. Section 1.5 in this chapter provides more details on DE. ACO was proposed to solve routing problems (Dorigo and Gambardella, 1997), and it has been used to solve job-shop scheduling, batch scheduling and combinatorial problems. ACO works on the principle of self-organization and transfer of information among individual ants through pheromones. Ants always search shortest
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
10
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
path between nest and available food. PSO mimics the social behavior of swarms (Kennedy and Eberhart, 1995); in this, particles or swarms search possible solutions in their neighborhood, and share their experiences with other particles. SA uses the concept of annealing (i.e., slow cooling) process in metallurgy (Kirkpatrick et al., 1983). In each iteration, a trial solution is generated in the neighborhood of the current solution, and the current solution is replaced by the trial solution if the latter has better objective value or satisfies Metropolis criterion to avoid trapping of search in local optimum regions. These steps are repeated many times before reducing the temperature. Tabu search (TS) maintains a short memory of trial solutions, and uses them to prohibit reverse moves (Glover, 1986) while searching for a better solution in the neighborhood. In the 21st century, several new metaheuristics were proposed. Geem et al. (2001) developed music inspired harmony search. Pham et al. (2005) proposed bees algorithm, which mimics food foraging behavior of honey bee colonies. Yang (2008) developed firefly algorithm, which is inspired by flashing behavior of fireflies. Yang and Deb (2009) introduced cuckoo search, which is inspired by obligate brood parasitism of some cuckoo species. 1.4 Multi-objective Optimization Problems and Their Solutions Many applications are likely to have two or more objective functions; such problems belong to MOO. For example, an MOO problem with three objectives is as follows. Objective functions: Decision variables: Constraints:
Minimize or maximize f1(x) Minimize or maximize f2(x) Minimize or maximize f3(x) x ≡ x1, x2… xn xL < x < xU g(x) < 0 h(x) = 0
(1.2a) (1.2b) (1.2c) (1.2d) (1.2e) (1.2f) (1.2g)
Here, f1(x), f2(x) and f3(x) are the given three objective functions. Solution of an MOO problem, having conflicting objectives, gives many optimal solutions, which are called Pareto-optimal front or nondominated solutions. See Bhaskar et al. (2000), Rangaiah (2009), Rangaiah and Bonilla-Petriciolet (2013) and Rangaiah et al. (2015) for
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Introduction
b2817-ch01
11
more details on MOO and its numerous applications in chemical engineering. Weighted sum and ε-constraint are two classical methods for converting an MOO problem into an SOO problem. In order to find the set of non-dominated solutions for MOO problem, the converted SOO problem is solved many times, obtaining one non-dominated solution each time. In the weighted sum method, a scalar weight is assigned to each objective function, and different values of weights are used to obtain different non-dominated solutions. The ε-constraint method optimizes the MOO problem for one objective function, while other objective functions are considered as additional inequality constraints in the SOO problem. Weighted sum and ε-constraint methods cannot accommodate preferred values for different objective functions (Deb, 2001; Rangaiah, 2009). Goal programming and compromise programming can accommodate such preference of the decision maker. In these, the desirable solution is that having the smallest difference between objective functions and their respective goals. Normal boundary intersection and normalized normal constraint are other methods to convert MOO problems into SOO problems. The converted SOO problem can be solved using a gradient-based method (e.g., GRG and SQP) or metaheuristics (e.g., GA, DE, ACO, PSO, SA and TS). All the metaheuristics (stochastic optimization methods) were proposed for solving SOO problems. Later, researchers adapted many of them to solve the MOO problem in a single run. MOO methods like nondominated sorting genetic algorithm-II (Deb, 2002), multi-objective differential evolution (Sharma and Rangaiah, 2013a; Chapter 5 in this book) and multi-objective particle swarm optimization (Coello and Salazar, 2002) can generate the complete Pareto-optimal front in a single run. ACO (Mariano and Morales, 1998), SA (Serafini, 1994; Ramteke and Gupta, 2009) and TS (Gandibleux et al., 1997) have also been adapted successfully for multiple objectives. 1.5 Differential Evolution Differential evolution (DE) was proposed by Storn and Price (1995, 1997) for solving SOO problems over continuous search space. Fig. 1.3 presents a simple pseudocode for classic DE. An initial population is randomly generated inside the bounds on decision variables, and values of objective functions and constraints are calculated for each and every
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
12
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
individual in the initial population. In each generation, a mutant individual is created for each target individual in the initial/current population, by using three or more other individuals randomly chosen from the initial/current population. After that, target and mutant individuals produce a trial individual by crossover operation. Finally, selection is performed between the target and trial individuals based on objective function value, to select the better individual for the subsequent generation. There are three parameters in classic DE algorithm: (1) population size (NP) with a recommended value of 5 to 20 times number of decision variables, (2) crossover probability (Cr), which can be between 0 and 1, and (3) mutation factor (F), which has a recommended value between 0 and 2. In addition, the user has to choose maximum number of generations (MNG) or iterations. See Section 1.7 for an illustration of multi-objective DE algorithm with a simple application. Specify DE parameters (population size, maximum number of generations, crossover probability and mutation factor) Initialize the population randomly inside the bounds on decision variables Evaluate objective function and constraints for all individuals in the initial population Generation loop For i = 1 to population size Select ith individual as target individual Generate a mutant individual using three random individuals in the population Generate trial individual using target and mutant individuals Calculate objective function and constraints for trial individual Selection between target and trial individuals, for the next generation Repeat the generation loop for maximum number of generations.
Fig. 1.3 A simple pseudocode for classic DE
Since its proposal in the year 1995, DE has undergone many changes and developments, and is now one of the popular and effective global optimization techniques. Its code and concepts are readily available on the internet (http://www.icsi.berkeley.edu/~storn/code.html). Researchers have improved classic DE in various aspects of the algorithm such as population initialization, mutation, crossover and selection operations (Price et al., 2005; Brest et al., 2006; Rahnamayan et al., 2008). A review
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Introduction
b2817-ch01
13
of these developments in DE until the year 2009 is available in Chapter 2 of this book. Recent advances in DE are summarized by Das et al. (2016). In particular, since DE performance for solving application problems depends on the values of its parameters, several studies have focused on the adaption of these parameters (Zhang and Sanderson, 2008; Huang et al., 2010; Wang et al., 2010). Further, DE has been successfully adapted for multiple objectives (Kukkonen and Lampinen, 2009; Dong and Wang, 2009; Gong and Cai, 2009; Gujarathi and Babu, 2010; Ali et al., 2012; Chen et al., 2014). Chapter 5 in this book describes one adaptation of DE for multiple objectives and an MS Excel-based program for it. Instead of commonly used maximum number of generations, new termination criteria have been proposed and studied in conjunction with DE (Srinivas and Rangaiah, 2007; Zhang and Rangaiah, 2011; Fernández-Vargas et al., 2016 for SOO; Sharma and Rangaiah, 2013c for MOO). 1.6 Applications of Differential Evolution in Chemical Engineering Chapter 2 in this book presents developments and applications of DE in chemical engineering, until the year 2009. Recently, Dragoi and Curteanu (2016) reviewed DE applications for solving chemical engineering problems. In order to provide an overview of DE applications in chemical engineering in recent years, Scopus database was searched using the keyword ‘differential evolution’ in the article title and keywords. Other criteria for narrowing the Scopus search are as follows: fields — article or review article, date range — January 2008 to April 2016, subject area — chemical engineering, energy and environmental science, language — English. In total, 904 articles were found, and 216 articles were short-listed based on their relevance to DE and chemical engineering. From these short-listed articles, Fig. 1.4 shows the number of journal papers on DE applications in chemical engineering and related areas over the years 2008 to 2016, and Table 1.2 summarizes number of articles published in different journals. It can be seen that nearly 70 papers are in energy-related journals (Table 1.2). DE applications in chemical engineering can be divided into two broad categories: (1) modeling or parameter estimation, and (2) process design and optimization. About 60% studies on DE applications in chemical engineering, energy and environmental science from the year
Differential Evolution in Chemical Engineering
14
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
40
No. of Journal Papers
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
2008 to April 2016 are on process design and optimization, and the remaining are on parameter estimation, mainly for estimating reaction kinetics. Most of the studies considered single objective function; only about 10% of studies considered two or more objective functions. Tables 1.3 and 1.4 summarize applications of DE in chemical engineering, and the commonly used objectives in these applications. Energy Conversion and Management journal has attracted the most number of DE studies (20 papers) followed by the Energy journal with 14 papers.
35 30 25 20 15 10 5 0 2008
2009
2010
2011
2012
2013
2014
2015
2016
Year of Publication Fig. 1.4 Number of reported DE applications in chemical engineering and related areas in recent years; number for 2016 is until April only
Some recent studies on DE applications have also considered modifications in DE algorithm aspects. Important modifications in DE algorithm are summarized below. The modified DE algorithms have been used for optimization of chemical engineering related applications. Adaptation of mutation factor, crossover probability and/or mutation strategy Population initialization, different sub-populations with different mutation strategies Termination criteria, equality and inequality constraints handling DE with PSO, DE with free search, DE with group search, DE with tabu list, DE with chaotic search, DE with sequential simplex, DE with clustering, DE with conjugate gradient method Permutation based DE, Discrete binary DE
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch01
15
Introduction
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Table 1.2 Number of reported DE applications in different journals (N - number of journal papers): January 2008 to April 2016 Name of Journal (J)
N
Adsorption Science and Technology AIChE J, Biochemical Engineering J, Bioprocess and Biosystems Engineering, Chemical Engineering Transactions, Energy and Fuels, International J of Heat and Mass Transfer, J of Environmental Chemical Engineering, J of Renewable and Sustainable Energy, Powder Technology, Water Science and Technology
1
Biotechnology and Bioprocess Engineering, Canadian J of Chemical Engineering, Chemical Industry and Chemical Engineering Quarterly, Computer Aided Chemical Engineering, Frontiers in Energy, J of Chemical and Engineering Data, J of Chemical Technology and Biotechnology, Korean J of Chemical Engineering, Petroleum Science and Technology, Renewable and Sustainable Energy Reviews, Separation and Purification Technology
2
Chemical Engineering and Technology, Chemical Engineering Communications, Chemical Engineering J, Ecological Modelling, Indian Chemical Engineer, International J of Energy Research, J of Petroleum Science and Engineering
3
Chemical Engineering Science, Chemical Product and Process Modeling, International J of Chemical Reactor Engineering, J of Chemical Engineering of Japan, J of Industrial and Engineering Chemistry, Renewable Energy
4
Asia-Pacific J of Chemical Engineering, Chemical Engineering Research and Design, Water Resources Research
5
Applied Thermal Engineering, Chemical Engineering and Processing: Process Intensification, J of the Taiwan Institute of Chemical Engineers
6
Computers and Chemical Engineering
8
Chinese J of Chemical Engineering, Fluid Phase Equilibria
9
Applied Energy
10
J of Natural Gas Science and Engineering
11
Industrial and Engineering Chemistry Research, International J of Hydrogen Energy
12
Energy
14
Energy Conversion and Management
20
Differential Evolution in Chemical Engineering
16
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah Table 1.3 Recent applications of DE in process modeling or parameter estimation
Reaction Engineering: Jet stirred reactor combustor, double moving beds coupled reactors, pyrolysis, industrial fluid catalytic cracking, 3-phase trickle bed reactor, batch and fed-batch fermentation, reactive system, ethylene oxide reactor, coal-water slurry gasifier, reaction network, CSTR, radial flow packed bed reactor, FT synthesis slurry reactor, reactive extraction, terephthalic acid production, ammonia synthesis.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Power Generation: Photovoltaic, power system, dispatch problem, wind power, solar cell/power, SOFC and PEM fuel cells, reheat regenerative power cycle Other Applications: Reservoir, cooling tower, Cr(VI) removal from wastewater, ionic liquids, biochemical system, partition coefficient estimation, solubility parameter estimation, PID controller tuning, phase equilibrium and stability, VLE modeling, model discrimination, soft sensor, slurry flow Common Objective Functions: Misfit, error, relative error, total relative error, absolute error, mean squared error, sum of squared error, root mean square error, mean absolute percentage error, integral error, integral time weighted absolute error, fitting error, Gibbs free energy, deviation from set point, error-in-variable, standard deviation of error, cross correlation of error.
Table 1.4 Recent applications of DE in process design and optimization Reactors: Fluidized and fixed bed reactors, methanol synthesis, CO2 conversion to methanol, dimethyl ether production, (packed bed) membrane reactor, thermally coupled reactors, thermally coupled dual membrane reactor, copolymerization reactor, naphtha reformer, naphtha pyrolysis, ethylene cracking furnace, alkylation process, Williams-Otto process, ethylene oxide reactor, Fischer-Tropsch reactor, cyclohexane dehydration, auto-thermal ammonia reactor, hydrogen production, styrene reactor Fermentation Processes: Fermentation, fed-batch fermentation, fermentationpervaporation and fermentation-extraction processes Separation Processes: Bixin extraction, filtering hydrocyclone, intensified distillation column, extractive distillation, middle vessel batch distillation, ethanol dehydration, solvent design for ethanol extractive fermentation, lactic acid production/recovery Power Generation: Distributed generation, electric power network, solar plant with storage, hydrothermal power system, supercritical CO2 Brayton cycles, renewable distributed generation, coal fired power plant, PEM fuel cell, power cycle with waste heat and LNG (liquefied natural gas) cold energy, solar thermal refrigeration systems Other Applications: Heat exchangers, HEN retrofitting, photovoltaic water pumping, water distribution, scheduling, blending process, pressure vessel design Common Objective Functions: Expected global cost, life cycle cost, total (annual) cost, profit, operating cost, efficiency, fuel cost, utility cost, primary energy consumption, emissions, production rate, purity, effectiveness, yield, productivity, conversion, recovery, selectivity, fugacity, activity, outlet mole fractions
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch01
17
Introduction
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
1.7 A Simple Application of DE A simple heat exchanger network (HEN), illustrated in Fig. 1.5, consists of one cold feed, one hot effluent stream and a heat exchanger (HE), a heater and a cooler. The cold feed stream enters at 30C (= TC,in) and it has to be heated to 125C (= TC,out) while the hot effluent stream enters at 115C (= TH,in) and it has to be cooled to 40C (= TH,out). The mass flow rate and heat capacity of the feed stream are respectively 16 kg/s and 4 kJ/(kg.K), and the mass flow rate and heat capacity of the effluent stream are respectively 20 kg/s and 3.8 kJ/(kg.K). Cooling Water o
TH,in = 115 C
T1
TH,out = 40oC
Effluent Heat Exchanger (HE)
Heater (QH) Feed TC,out = 125oC
T2
Cooler (QC) TC,in = 30oC
Steam Fig. 1.5 Schematic of a heat exchanger network (HEN) for heat recovery and reuse number
HE in Fig. 1.5 helps to recover and reuse thermal energy, thus reducing both steam required in the heater and cooling water required in the cooler. This reduces the operating cost. However, inclusion of HE is likely to increase the required investment (also known as capital cost). Thus, there will be a trade-off between operating cost and investment, which is the case in many engineering applications. The design question is whether HE should be included and, if so, what should be the size of HE. This can be formulated as an optimization problem with one or two objectives, and DE can be used to solve it. This section describes the formulated optimization problem with two objectives, and then presents results obtained using IMODE program (described in Chapter 5), which is based on multi-objective DE and is implemented in MS Excel environment. Note that optimization problem formulation requires background and knowledge in the relevant discipline. For example, HEN
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
18
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
problem formulation (described below) is simple and easy to understand by chemical and mechanical engineers. HE, cooler and heater in Fig. 1.5 are all assumed to be counter-current type, which is the most common. Temperatures of the effluent and feed streams after HE are denoted as T1 and T2 respectively. If there is no HE, then T1 = TH,in and T2 = TC,in. HEN under consideration is at the design stage, and so the area of the heat exchanger, heater and cooler (denoted by AHE, AH and AC respectively) can be varied for optimization. The governing (model) equations and the optimization problem for the HEN can be developed based on heat transfer principles and suitable cost correlations/data for the investment and operating cost. One of the governing equations is the energy balance, which, for HE, is given by: 16 × 4 × T − T
= 20 × 3.8 × (T
,
,
−T )
(1.3)
The above equation can be re-arranged to find T as follows: T =T
+
,
× . ×
× T
−T
,
(1.4)
The two objective functions are: Minimize Investment ($) = IH + IHE + IC Minimize Operating Cost ($/year) = CSteam + CCW
(1.5a) (1.5b)
Here, IH, IHE and IC are respectively the investment for the heater, HE and cooler. Each of them is given by: Ii = 38000 + 520Ai0.9
for i = Heater, HE and Cooler
(1.6)
Here, Ai is the heat transfer area of equipment i, which is given by the following equations obtained from the heat transfer rate equation involving log mean temperature difference. A A A
= =
× ×(
(
,
× . ×(
,
,
)×
,
(1.7a)
,
)×
,
,
=
)
(1.7b)
,
× . ×(
,
)×
(
)
,
,
(1.7c)
In the above equations, overall heat transfer coefficients for heater, HE and cooler are assumed to be UH (= 0.78 kW/m2-K), UHE (= 0.50 kW/ m2-K) and UC (= 0.50 kW/m2-K), respectively. Further, saturated steam
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch01
19
Introduction
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
(heating medium) enters the heater at 180oC and leaves as condensate at the same temperature, thus giving the latent heat of condensation for heating purpose. Cooling water enters and exits the cooler at 30oC and 40oC, respectively, due to heat transferred from the effluent. In the operating cost equation (1.5b), steam cost (CSteam) and cooling water cost (CCW) are given by: CSteam = 400 × 16 × 4 × (T , − T ) CCW = 25 × 20 × 3.8 × (T − T , )
(1.8a) (1.8b)
Here, utility costs are taken to be 400 US$/(kW.y) for steam and 25 US$/(kW.y) for cooling water. In equations 1.3 to 1.8, TH,in = 115oC, TH,out = 40oC, TC,in = 30oC and TC,out = 125oC from the problem statement. Further, knowing T1, all other unknown quantities in these equations can be calculated (i.e., T2 from equation 1.4, Ai from equation 1.7, investment from equation 1.6 and operating cost from equation 1.8). Hence, HEN optimization in Fig. 1.5 has only one decision (independent) variable, namely, T1. Lower bound for this variable is TH,out = 40oC, which still ensures a minimum approach temperature of 5oC, and upper bound is 115oC (i.e., TH,in in the absence of HE). In addition, the calculated/dependent variable, T2 is also constrained between 30oC (in the absence of HE) and 110oC (= 115 - 5 to ensure a minimum approach temperature of 5oC). In summary, the optimization problem is simultaneous minimization of two objectives in equation 1.5 with respect to T1 between 40 and 115oC subject to two inequality constraints to keep T2 between 30 and 110oC. Note that equations 1.4, 1.6, 1.7 and 1.8 are essentially equality constraints but they can be solved easily one by one. The latter approach is better since stochastic optimizers are not effective in handling equality constraints (Sharma and Rangaiah, 2013b). HEN optimization problem is entered into the Objectives & Constraints worksheet of IMODE program (Fig. 1.6a), and it is linked with the Main Program Interface worksheet (Fig. 1.6b). Algorithm parameters are the default values along with population size of 40 and maximum number of generations of 100. HEN design problem is solved by clicking ‘Run IMODE’ icon. Optimal values of objectives (Investment and Operating Cost), decision variable (T1) and two inequality constraints on T2 are plotted in Fig. 1.7. This figure also contains plots showing the variation of Operating Cost with area of heater, heat exchanger and cooler, respectively.
Differential Evolution in Chemical Engineering
20
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
b2817-ch01
S. Sharma and G.P. Rangaiah A
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
9in x 6in
B C D E F G H Worksheet for Providing the Decision Variables, Functions and Constraints (Scroll down for instructions on calculation and linking of objectives and constraints.) HEN example Decision Variable and its Bounds Description Value T1 60 Objective Investment Operating Cost Calculated Values Quantity T2 LMTD of Heater A of Heater LMTD of HE A of HE LMTD of Cooler A of Cooler Invesment for Heater Investment for HE Investment for Cooler Cost of Steam Cost of CW
I
J
K
Lower Bound Upper Bound 40 115 Constraints T2 > T2 <
k$ k$/year
290 798.0
30 110
Value 95.31 68.78 35.42 24.48 341.46 14.43 210.72 50891 137086 102170 760000 38000
Objectives & Constraints
MOOSetup
Intermediate Results
Results at ChiTC
Results at SSTC
Results after MNG
Fig. 1.6a HEN optimization problem entered into the Objectives & Constraints worksheet of the IMODE program A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
B
C
D
E
F
H
G
I
J
K
L
M
N
Main Program Interface Algorithm: Objective Functions: Value Cell Goal NumF Design Variables: Value Cell Minimum Maximum Type of Variables NumX Inequality Constraints: Set Cell Compare Type Constraint Limit NumC
IMODE (Improved Multi-Objective Differential Evolution) for problems with continuous/integer variables and inequality constraints Add Objective Functions
Add Decision Variables
F1 F2 290.15 798 Minimize Minimize
T1 88.06 40 115 Continuous
1 Add Constraints
Algorithm Parameters: NP Cr
T2 95.31 Greater 30
T2 95.31 Lesser 110
MNG F
100 0.5
ACooler AHeater AHE 35.42 341.46 210.72 Greater Greater Greater 0 0 0
5
40 0.5
Generation Interval: Saving Intermediate Result 20 (This should be less than the maximum number of generations) Objectives & Constraints
MOOSetup
20 0.01
TLS TR
δGD δSP
Run IMODE
Intermediate Results
Results at ChiTC
0.0003 0.1
SSTC Parameters 0.1 TC MNG λ1 0.1 λ2 0.1 λ3 0.9 Rcrit
Results at SSTC
Results after MNG
Fig. 1.6b Main program interface of the IMODE program for the HEN optimization problem
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Introduction
b2817-ch01
21
As expected and as can be seen in Fig. 1.7, Investment and Operating Cost objectives are conflicting. More importantly, MOO provides quantitative trade-off between Investment and Operating Cost, and also many optimal solutions along with the optimal values of decision variable, constraints and dependent variables (namely, areas of heater, heat exchanger and cooler). All these give a deeper insight into the process on hand and for selecting one of the optimal solutions based on the preferences of the decision maker and other considerations. Before using MOO results, it is desirable to analyze and explain qualitatively the trends of objectives, decision variables and other quantities, in order to ensure their validity. For example, increasing T1 leads to increased energy recovery in the heat exchanger, thus resulting in the decreased requirement of steam (in the heater) and cooling water (in the cooler), which reduces Operating Cost. In terms of equipment size, heat exchanger area increases, and areas of the heater and cooler decrease as T1 increases. These variations are consistent with the expectations based on the knowledge in the heat transfer field. The overall outcome of area changes is the increased investment cost. Application described in this section, although realistic, is relatively simple. More complex and realistic applications are described and discussed in many chapters of this book. For example, IMODE program and its application are presented in Chapter 5, retrofitting of large HENs is covered in Chapter 6, and Chapter 10 describes optimization of bioethanol separation by the hybrid process of distillation and vapor permeation. The next section outlines scope and contributions of all chapters in this book. 1.8 Scope and Outline of Chapters After this Introduction chapter in Part I Overview, the subsequent chapters in this book are organized in Parts II and III. Chapters 2 to 5 form Part II on DE developments, and Chapters 6 to 13 in Part III cover many chemical engineering applications of DE. An outline of these chapters is presented in this section. Many of the chapters contain exercises at the end for practice by the interested readers. Moreover, these exercises can also be adapted as projects for students in optimization courses. In Chapter 2, Chen et al. describe DE, its parameters and their values. They summarize the proposed modifications to various components of
Differential Evolution in Chemical Engineering
22
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
Min. Operating Cost (K$/year)
2500 2000 1500 1000 500 0 230
270 310 350 390 Min. Investment (k$)
430
40
55
70 85 T1 (ºC)
100
115
Min. Operating Cost (k$/year)
3000 2500 2000 1500 1000 500 0 30 40 50 60 70 80 90 100 110 T2 (ºC)
10 20 30 40 50 60 70 80 90 Area of Heater (m2)
3000 Min. Operating Cost (k$/year)
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
3000
2500 2000 1500 1000 500 0 0
200
400
600
800 1000 1200 2
Area of Heat Exchanger (m )
100 150 200 250 300 350 400 Area of Cooler (m2)
Fig. 1.7 Pareto-optimal front of Operating Cost and Investment in the top left plot, and the corresponding decision variable and dependent variables in other plots
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Introduction
b2817-ch01
23
DE; these modifications have improved DE capabilities for tackling a variety of optimization problems. Further, Chen et al. provide an overview of chemical engineering applications of DE reported until the year 2009. In particular, DE has found many applications for parameter estimation and modeling in addition to process design and operation. Recently, Dragoi and Curteanu (2016) reviewed DE applications for solving chemical engineering problems. Given these developments and applications, DE is attractive and useful both as a simple general optimizer and as a sophisticated tool to solve complex chemical engineering applications. Chapter 3 by Rahimpour and Hamedi provides an overview on DE applications in chemical reaction engineering. DE is a simple and robust technique suitable for optimization of nonlinear and complicated models resulting from mass and energy balances combined with intricate thermodynamic and other auxiliary equations. Considerable effort has been made to optimize reactor operating conditions using DE. In Chapter 3, DE algorithm is first explained, and then different aspects of reactor and kinetic modeling are introduced. Afterwards, key decision variables and objective functions are discussed. Based on previous publications, the decision variable section is divided into temperature, pressure, flow rate, membrane thickness, reactor size, feed concentration and kinetic parameters. Chapter 3 summarizes the main aspects of using DE in chemical reaction engineering, thus providing an overview for future researchers. Srinivas and Rangaiah describe two versions of DE with a Tabu List (DETL) in Chapter 4. One version incorporates the concept of avoiding revisits during the search, using tabu check in the generation step of DE. Another version implements the same in the evaluation step of DE. These versions are evaluated on benchmark and phase stability problems. Benchmark problems consist of 2 to 20 decision variables and a few to hundreds of local minima whereas phase stability problems involve multiple components and comparable minima. Further, a new benchmark problem with characteristics similar to phase stability problems is proposed and used. The results show that the performance of the two versions of DETL is comparable, and it is better than DE in number of function evaluations and better than TS in reliability. Chapter 5 by Sharma et al. presents the Integrated Multi-Objective Differential Evolution (IMODE) program in MS Excel, useful for solving MOO problems. The algorithm in this program has four main
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
24
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
parts: multi-objective DE, tabu list for avoiding revisit of search space, self-adaptation of DE parameters, and two search termination criteria besides maximum number of generations. All these features of the IMODE program make it reliable and efficient for solving engineering optimization problems. To illustrate the application of this program, amine absorption process, commonly used to remove acid gases from the natural gas, is simulated in Aspen HYSYS and then optimized using the IMODE program for two objectives: capital and operating costs. Finally, performance of the two improvement-based termination criteria in the IMODE program is compared using the multi-objective performance metrics. Heat Exchanger Network (HEN) retrofitting, the application in Chapter 6 by Sreepathi et al., improves the energy efficiency of current processes by reducing utilities required through suitable changes. It is complicated involving many integer and continuous variables, and numerous combinations. In this chapter, a Multi-Objective DE (MODE) program is developed in R, and then applied to four case studies on HEN retrofitting. Results obtained by this program are compared with those using the NGPM program, based on the elitist non-dominated sorting genetic algorithm, in MATLAB. Results show that the MODE program gives better solutions than NGPM for HEN retrofit problems tested. In Chapter 7, Bonilla-Petriciolet et al. apply DE and TS, each along with a local optimizer, to phase stability and equilibrium calculations in reactive systems, which are formulated using transformed composition variables. This study shows, for the first time, that both DE and TS are successful for solving phase stability and equilibrium calculations in multi-component and multi-reactive systems using transformed composition variables. Also, Bonilla-Petriciolet et al. demonstrate the use of performance profiles for tuning algorithm parameters. Results show that DE is better than TS for the applications tested, but requires more computational effort. For both phase stability and equilibrium calculations, maximum number of generations significantly affects the performance of these stochastic methods, and hence a suitable termination criterion should be selected to improve the performance. Design and optimization of all possible alternatives for the separation of multicomponent mixtures by distillation could be too time consuming or simply not possible. The systematic synthesis methodology, proposed in Chapter 8 by Errico et al., allows the definition of a search space of alternatives, where the simple column sequences are directly related to
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Introduction
b2817-ch01
25
all the alternatives. When this correspondence is extended to the distillation column design, it is possible to define ‘Sequential Design Method’. This method together with the multi-objective DE algorithm, results in a successful methodology to obtain the optimal design in reasonable computation time avoiding the evaluation of all alternatives included in the search space. In Chapter 9, Sánchez-Ramírez et al. optimize intensified separation processes using DE with tabu list. Despite many advantages, optimization of intensified processes is challenging since the used models are highly non-linear with continuous and discrete variables. DE with tabu list has been successful for solving this kind of complex problems for one or several objective functions. Hence, this chapter illustrates its capabilities for intensified separation processes considering two examples. One example is reactive distillation for manufacturing diphenyl carbonate as a green product alternative, minimizing both the total annual cost and the condition number obtained from control properties. Another example is separation of acetone/butanol/ethanol/ water mixture in the fermentation broth. This separation using a dividing wall column is optimized for minimizing the total annual cost, ecoindicator 99 and the condition number. Chapter 10 by Singh and Rangaiah is on DE application to bioethanol recovery and dehydration by Distillation and Vapor Permeation (DVP). Bioethanol obtained from biomass fermentation is dilute and unsuitable for use in automobiles. For recovery (pre-concentration) and dehydration (purification) of bioethanol from the fermentation broth, distillation is widely used. Membrane separation is increasingly promising for bioethanol separation. Hence, this chapter presents the development of a hybrid DVP process to produce fuel-grade (99.8 wt.%) ethanol, and then MOO of DVP process using DE. Pareto-optimal solutions for minimizing GHG and cost of manufacturing are presented and discussed. Manufacturing cost of separation by DVP process is 0.1829 $/(kg of bioethanol) with 99.8% recovery of bioethanol. Chapter 11 by Koop et al. reports the application of DE to optimize the performance of a fermentation process for xylitol production by Candida mogii yeast using a fed-batch reactor. A dynamic model that assumes the growth of cells without mortality rate was used; it has been validated earlier via experimental studies on a laboratory-scale unit. Different operating scenarios for this fermentation process have been considered, and the impact of the decision variables has been analyzed.
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
26
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
Numerical performance of DE has been compared with the results obtained using Particle Swarm Optimization (PSO) and Artificial Bee Colony (ABC). This comparison shows that DE is a reliable method for dynamic optimization of xylitol production in a fed-batch fermentation process, and it can outperform both PSO and ABC. In Chapter 12, Wang investigates nested DE for mixed-integer bilevel optimization in genome-scale metabolic networks. Many bi-level optimization methods have been used to determine optimal strain designs for the genome-scale metabolic networks of bacteria. Such bi-level optimization problems are generally reduced to single-level problems using strong duality theory. This approach can exponentially increase computation time, and does not guarantee that a growth-coupled production strain would be obtained. Chapter 12 introduces an equality constraint to minimize and maximize flux variability in the strain design problem, which guarantees a growth-coupled strain. A nested hybrid DE algorithm is proposed for solving the constrained optimization problem to obtain a set of growth-coupled production strains. It is tested through the simulation of the iAF1260 metabolic network of E. coli. The last chapter in this book, Chapter 13 by Dragoi and Curteanu focuses on DE applications for modeling and optimization in polymerization reaction engineering. First, difficulties in the modeling polymerization processes are presented to justify the use of Artificial Neural Networks (ANN) and DE. These difficulties are due to complexity of the reaction medium, lack of complete knowledge of reaction mechanism, problems in developing and solving phenomenological models, their accuracy and/or potential for inclusion of on-line control procedures. Neuro-evolutive techniques are recommended for modeling and optimizing such complex polymerization processes. A section is dedicated to general aspects of using DE in combination with ANN for developing optimal neural models and for determining optimal operating conditions. Two applications: synthesis of polyacrylamide based hydrogels and of siloxane-siloxane copolymers, are discussed in detail. 1.9 Conclusions
Systematic and thorough optimization is essential in chemical engineering and many other fields for efficient and sustainable industries producing goods needed by the modern society. Stochastic optimization methods, also known as metaheuristics, are
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Introduction
b2817-ch01
27
attractive because of their applicability to any type of problems irrespective of discontinuities, nonlinearities and derivative availability. Of these, DE proposed by Storn and Price in 1995 is a simple and effective technique for both global optimization and MOO. Since its proposal, DE has undergone many changes and developments, and its performance and applicability have been improved. Its code and concepts are readily available on the internet (http://www.icsi.berkeley.edu/~storn/code.html). DE developments and applications to chemical engineering are reviewed in Chapter 2 of this book and in Dragoi and Curteanu (2016). Recent advances in DE are summarized by Das et al. (2016). A number of computer programs for DE are available at the website mentioned earlier in this paragraph. MS-Excel based program for MOO is described in Chapter 5; it is based on DE, and incorporates parameter adaptation and performance-based termination criteria besides the common maximum number of generations. It can be used for both SOO (by putting a constant value to the second objective) and MOO by engineers and scientists from any field. Many chapters in this book describe important applications of DE to chemical engineering application, in detail. These include phase stability and equilibrium calculations, retrofitting heat exchanger networks, amine absorption process for CO2, distillation sequences, intensified separation processes, distillationvapor permeation process for bioethanol, fermentation process for xylitol, metabolic networks and polymerization reactions. DE developments and applications in chemical engineering covered in this book will be useful to both researchers and practitioners. Many chapters in this book have exercises at the end for practice by interested readers. Several useful programs are available at the book website or by contacting the relevant contributor of the chapters. After going through this Introduction chapter, each chapter can be read and used on its own. The editors hope that each of the chapters in this book will benefit many
Differential Evolution in Chemical Engineering
28
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
readers and will lead to further developments and applications of DE.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Exercises
1.1. Describe a scenario requiring optimization. The scenario can be in chemical engineering, any other field or daily life (e.g., career plan of a graduate student). In this scenario, identify and state the objectives, decision variables and constraints, qualitatively or quantitatively. 1.2. Solve the HEN problem in Section 1.7, and reproduce the presented results using the IMODE program (described in Chapter 5 in this book) or any other MOO program. Analyze the sensitivity of the non-dominated solutions in Section 1.7 with respect to ±10% changes in the mass flow rates and temperatures of feed and effluent. Which of these variables has the significant effect on the non-dominated solutions? 1.3. An important process in petroleum refining is the alkylation process, whose product is used for blending with refinery products such as gasoline and aviation fuel in order to increase their Octane Number. See Rangaiah (2009) for more details on alkylation process. Bi-objective optimization problem of alkylation process is as follows. Maximize Revenue, R ($/day) = 0.063x4x7 Minimize Cost, C ($/day) = 5.04x1+0.035x2+10.0x3+3.36x5 With respect to
(1.9a) (1.9b)
x1, x7 and x8
Subject to 0 x1 2,000 90 x7 95 3 x8 12 0 [x2 x1 x8 – x5] 16,000 0 [x3 0.001 (x4 x6 x9)/(98-x6)] 120 0 [x4 x1(1.12 + 0.13167x8 – 0.006667x82)] 5,000 0 [x5 1.22x4 – x1] 2,000
(1.9c) (1.9d) (1.9e) (1.9f) (1.9g) (1.9h) (1.9i)
Differential Evolution in Chemical Engineering
9in x 6in
Introduction
b2817-ch01
29
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
85 [x6 89 + (x7 - (86.35 + 1.098x8 – 0.038x82))/0.325] 93 (1.9j) 1.2 [x9 35.82 – 0.222x10] 4 (1.9k) 145 [x10 – 133 + 3x7] 162 (1.9l) In this problem, the 7 inequality constraints in equations 1.9f to 1.9l are the bounds on the 7 variables (x2, x3, x4, x5, x6, x9 and x10) in the original problem, and they arise from the elimination of these variables from the 7 equality constraints in the model, thus making them dependent variables. Solve the above problem using IMODE program (described in Chapter 5) or any other MOO program. Discuss the trend of values of objectives, decision variables and dependent variables of the nondominated solutions obtained. References Ali M., Siarry P. and Pant M. An efficient differential evolution based algorithm for solving multi-objective optimization problems, European Journal of Operational Research, 217, pp. 404-416, 2012. Bhaskar V., Gupta S. K. and Ray A. K. Applications of multi-objective optimization in chemical engineering, Reviews in Chemical Engineering, 16(1), pp. 1-54, 2000. Brest J., Greiner S., Boskovic B., Mernik M. and Zumer V. Self-adaptive control parameters in differential evolution: a comparative study on numerical benchmark problems, IEEE Transactions on Evolutionary Computation, 10(6), pp. 646-457, 2006. Chen X., Du W. and Qian F. Multi-objective differential evolution with ranking-based mutation operator and its application in chemical process optimization, Chemometrics and Intelligent Laboratory System, 136, pp. 85-96, 2014. Coello Coello C. A. and Salazar Lechuga M. MOPSO: a proposal for multi-objective particle swarm, IEEE Congress on Evolutionary Computation, pp. 1051-1056, 2002. Das S., Mullick S. S. and Suganthan P.N. Recent advances in differential evolution — an updated survey, Swarm and Evolutionary Computation, 27, pp. 1-30, 2016. Deb K. Multi-objective optimization using evolutionary algorithm, John Wiley & Sons, Chichester, UK, 2001. Deb K., Pratap A., Agarwal S. and Meyarivan T. A fast and elitist multi-objective genetic algorithm: NSGA-II, IEEE Transactions on Evolutionary Computation, 6(2), pp. 182197, 2002. Dong N. and Wang Y. Multi-objective differential evolution based on opposite operation, IEEE Conference on Computational Intelligence and Security, pp. 123-127, 2009. Dorigo M. and Gambardella L. M. Ant colony system: a cooperative learning approach to the traveling salesman problem, IEEE Transactions on Evolutionary Computation, 1(1), pp. 53-66, 1997. Dragoi E. N. and Curteanu S. The use of differential evolution for solving chemical engineering problems, Review in Chemical Engineering, 32(2), pp. 149-180, 2016. Edgar T. F., Himmelblau D. M. and Lasdon L. S. Optimization of chemical processes, 2nd ed., McGraw-Hill, 2001.
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
30
9in x 6in
b2817-ch01
S. Sharma and G.P. Rangaiah
Fernández-Vargas J. A., Bonilla-Petriciolet A., Rangaiah G. P., and Fateen S. K. Performance analysis of stopping criteria of population-based metaheuristics for global optimization in phase equilibrium calculations and modeling, Fluid Phase Equilibria, 427, pp. 104-125, 2016. Floudas C. A. Deterministic global optimization: theory, methods and applications, 3rd ed., Springer, 2013. Gandibleux X., Mezdaoui N. and Freville A. A tabu search procedure to solve multiobjective combinatorial optimization problems, In: Caballero R., Ruiz F. and Steuer, R. (editors), Advances in multiple objective and goal programming, Springer Verlag, pp. 291-300, 1997. Geem Z. W., Kim J. H. and Loganathan G. V. A new heuristic optimization algorithm: harmony search, Simulation, 76(2), pp. 60-68, 2001. Glover F. Future paths for integer programming and links to artificial intelligence, Computers and Operations Research, 13(5), pp. 533-549, 1986. Gong W. and Cai Z. An improved multi-objective differential evolution based on Paretoadaptive ε-dominance and orthogonal design, European Journal of Operation Research, 198(2), pp. 576-601, 2009. Gujarathi A. M. and Babu B. V. Hybrid multi-objective differential evolution (H-MODE) for optimization of polyethylene terephthalate (PET) reactor, International Journal of Bio-Inspired Computation, 2(3-4), pp. 213-221, 2010. Himmelblau D. M. Applied non-linear programming, McGraw-Hill, 1972. Huang V. L., Zhao S. Z., Mallipedi R. and Suganthan P. N. Multi-objective optimization using self-adaptive differential evolution algorithm, IEEE Congress on Evolutionary Computation, pp. 191-195, 2009. Kennedy J. and Eberhart R. C. Particle swarm optimization, IEEE International Conference on Neural Networks, pp. 1942-1948, 1995. Kirkpatrick S., Gelatt C. D. and Vecchi M. P. Optimization by simulated annealing, Science, 220, pp. 671-680, 1983. Kukkonen S. and Lampinen J. Performance assessment of generalized differential evolution 3 with a given set of constrained multi-objective test problems, IEEE Congress on Evolutionary Computation, pp. 1943-1950, 2009. Mariano C. E. and Morales E. A multiple objective ant-Q algorithm for the design of water distribution irrigation networks, Technical Report HC-9904, Mexican Institute of Technology of Water, 1999. Masuduzzaman and Rangaiah G. P. Multi-objective optimization applications in chemical engineering, In: Rangaiah, G. P. (editor), Multi-objective optimization: techniques and applications in chemical engineering, World Scientific, Singapore, 2009. Pham D. T., Ghanbarzadeh A., Koc E., Otri S., Rahim S. and Zaidi M. The bees algorithm, Manufacturing Engineering Centre, Cardiff University, 2005. Price K. V., Storn R. M. and Lampinen, J. A. Differential evolution - a practical approach to global optimization, Springer, Berlin/Heidelberg, 2005. Rahnamayan S., Tizhoosh H. R. and Salama M. M. A. Opposition versus randomness in soft computing techniques, Applied Soft Computing, 8(2), pp. 906-918, 2008. Ramteke M. and Gupta S. K. Multi-objective genetic algorithm and simulated annealing with the jumping gene adaptations, In: Rangaiah, G. P. (editor), Multi-objective optimization: techniques and applications in chemical engineering, World Scientific, Singapore, 2009.
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Introduction
b2817-ch01
31
Rangaiah G. P. Multi-objective optimization: techniques and applications in chemical engineering, World Scientific, Singapore, 2009. Rangaiah G. P. Stochastic Global Optimization: Techniques and Applications in Chemical Engineering, Singapore, 2010. Rangaiah G. P. and Bonilla-Petriciolet A. (Editors), Multi-Objective Optimization in Chemical Engineering: Developments and Applications, John Wiley, 2013. Rangaiah G. P., Sharma S. and Sreepathi B. K. Multi-objective optimization for the design and operation of energy efficient chemical processes and power generation, Current Opinion in Chemical Engineering, 10, pp. 49-62, 2015. Ravindran A., Raqsdell K. M. and Reklaitis G. V. Engineering optimization: methods and applications, 2nd ed., John Wiley & Sons, 2006. Serafini P. Simulated annealing for multi-objective optimization problems, Multiple Criteria Decision Making, pp. 283-292, 1994. Sharma S. and Rangaiah G. P. Multi-objective optimization applications in chemical engineering, In Rangaiah G. P. and Bonilla-Petriciolet A. (editors), Multi-Objective Optimization in Chemical Engineering: Developments and Applications, Wiley, 2013a. Sharma S. and Rangaiah G. P. Improved constraint handling technique for multiobjective optimization with applications to two fermentation processes, In Rangaiah G. P. and Bonilla-Petriciolet A. (editors), Multi-Objective Optimization in Chemical Engineering: Developments and Applications, Wiley, 2013b. Sharma S. and Rangaiah G. P. An improved multi-objective differential evolution with a termination criterion for optimizing chemical processes, Computers and Chemical Engineering, 56, pp. 142-154, 2013c. Srinivas M. and Rangaiah G. P. Differential evolution with tabu list for global optimization and its application to phase equilibrium and parameter estimation problems, Industrial and Engineering Chemistry Research, 46, pp. 3410-3421, 2007. Storn R. and Price K. Differential evolution - a simple and efficient adaptive scheme for global optimization over continuous spaces, Technical Report TR-95-012, ICSI, March 1995. Storn R. and Price K. Differential evolution - a simple and efficient heuristic for global optimization over continuous space, Journal of Global Optimization, 11(4), pp. 341-359, 1997. Yang X. S. and Deb S. Cuckoo search via Lévy flights, Proceedings of world congress on nature and biologically inspired computing (NaBIC 2009), IEEE Publications, USA, pp. 210-214, 2009. Yang X. S. Nature-Inspired Metaheuristic Algorithms, Luniver Press (2008). Wang Y. N., Wu L. H. and Yuan X. F. Multi-objective self-adaptive differential evolution with elitist archive and crowding entropy based diversity measure, Soft Computing, 14, pp. 193-209, 2010. Zhang H., Rangaiah G. P. and Bonilla-Petriciolet A. Integrated differential evolution for global optimization and its performance for modeling vapor-liquid equilibrium data, Industrial and Engineering Chemistry Research, 50, pp. 10047-10061, 2011. Zhang J. and Sanderson A. C. Self-adaptive multi-objective differential evolution with the directional information provided by archived inferior solutions, IEEE Congress on Evolutionary Computations, pp. 2801-2810, 2008.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
April 20, 2017
17:3
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch02
Chapter 2
Differential Evolution: Method, Developments and Chemical Engineering Applications Shaoqiang Chen, Gade Pandu Rangaiah∗ and Mekapati Srinivas Department of Chemical and Biomolecular Engineering National University of Singapore, 117585 Singapore ∗ Corresponding
author:
[email protected]
Abstract This chapter1 describes differential evolution (DE), its parameters and their values. Then, proposed modifications to various components of DE algorithm until the year 2009 are summarized; these modifications have improved DE capabilities for tackling a variety of optimization problems. Finally, chemical engineering applications of DE reported until the year 2009 are reviewed. In particular, DE has found many applications for parameter estimation and modeling in addition to process design and operation. Further developments and applications of DE since the year 2009 are outlined in Chapter 1. Given these developments and applications, DE is
1 This chapter is reproduced from Rangaiah G.P. (Editor), Stochastic Global Optimization:
Techniques and Applications in Chemical Engineering, Vol. 2 in the Advances in Process Systems Engineering, World Scientific, Singapore, 2010.
35
page 35
April 20, 2017
17:3
36
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch02
S. Chen, G.P. Rangaiah and M. Srinivas
attractive and useful both as a simple general optimizer and as a sophisticated tool to solve complex chemical engineering applications.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Keywords: Differential Evolution, Chemical Engineering Applications, Parameter Estimation, Process Design, Process Operation.
2.1 Introduction In global optimization problems where little knowledge of the problem is available (i.e. black-box problems), stochastic optimization methods are preferred due to their ability to escape local optima. One of these techniques, pioneered by Price and Storn since 1995, is the method of differential evolution (DE), which has received much attention over the years. DE was first developed as a tool to solve the Chebychev polynomial fitting problem (Price et al., 2005). With parameters (i.e. decision variables) of grossly different magnitudes and multiple local minima, this problem is challenging for most general purpose optimizers. DE, however, was able to tackle the Chebychev polynomial fitting problem with ease. This finding generated much interest in this method, and numerous developments have been made to DE over the years. Like other stochastic methods, it does not require a priori information about the system to be solved, and can be deployed for solving most optimization problems. DE is simple to understand, and the parameters involved are easy to manipulate; it has relatively fast convergence and its success in finding the global optima for many known problems has been more than encouraging (Price et al., 2005). In this chapter, DE is described in the next section, along with a brief discussion on tuning its parameters; both a flowchart and pseudo code for DE are presented for the benefit of readers new to DE. Then, modifications and enhancements made to DE by several researchers are reviewed; this section is useful to researchers considering further enhancements to DE. Finally, reported applications of DE in Chemical Engineering are summarized for the benefit of researchers and practitioners to solve Chemical Engineering problems. The chapter ends with concluding remarks on DE, its developments and applications.
page 36
April 20, 2017
17:3
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch02
DE: Method, Developments and Applications
37
2.2 Description of DE
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
An optimization problem with a single objective, f is as follows: minimize f (x 1 , x 2 , . . . , x N ) with respect to x 1 , x 2 , . . . , x N subject to ln ≤ x n ≤ u n for n = 1, 2, . . . ., N
(2.1)
Here, N is the number of decision variables; ln and u n are respectively the lower and upper bound on x n , and they take large negative and positive value respectively for unbounded problems. In addition, an optimization problem may contain inequality constraints (namely, gk for k = 1, 2, . . . , K ) and equality constraints (namely, h m for m = 1, 2, . . . , M). The bounds on decision variables and constraints define the feasible space for optimization search. For simplicity, only unconstrained and unbounded problems are considered in the subsequent description of DE. If there are bounds and/or constraints, they have to be handled by a suitable technique; for example, constraints can be handled by the penalty function technique. Note that decision variables (x n ) are referred as “parameters” in parameter estimation problems. Price et al. (2005) employ parameters instead of variables in their book devoted to DE, perhaps, in view of the origin of DE for parameter estimation in Chebyshev polynomials. However, for consistency throughout this book and with many other books on optimization, we use decision variables (or, simply, variables) instead of “parameters” in this chapter. DE is both simple (see the flowchart and pseudo code in Figs. 2.1 and 2.2 respectively) and natural (Price et al., 2005). It has its basis in Darwin’s natural selection theory, and is similar to genetic algorithms (GAs) except for one important factor: several GAs, particularly earlier versions, encode decision variables as bit strings whereas DE encodes them as floating-point numbers. Hence, GAs are more suited for combinatorial problems, while DE is more adapted to continuous problems. A flowchart of the classic DE algorithm is shown in Fig. 2.1, and the main steps of this algorithm are as follows. Initial population: DE uses a population of N p individuals or points in the search space. Randomly initialize N p N-dimensional vectors over the
page 37
April 20, 2017
17:3
Differential Evolution in Chemical Engineering
38
9in x 6in
b2817-ch02
S. Chen, G.P. Rangaiah and M. Srinivas Initialization: Generate Np population vectors over the specified search space. Initialize Generation = 0.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Set index, j = 1 for the first target vector in the population.
Mutation: Select 3 different vectors randomly (other than the target vector) from the population. The difference vector between the last two vectors is multiplied by the mutation factor, F and added to the first (base) vector to give the mutant vector. Crossover: For target vector j, a trial vector, with each variable value coming from either the mutant vector or the target vector, is created. The probability of a variable value coming from the mutant vector is Cr.
Selection: Either the target or the trial vector, whichever has better objective function value is chosen to enter the next generation. j = j+1
No Is j = Np? Yes Generation = Generation + 1
No
Convergence Criterion/a Satisfied? Yes Print Results and Stop.
Fig. 2.1 Flow chart for differential evolution; see the pseudo code in Fig. 2.2 for equations for mutation, crossover and selection operations
page 38
April 20, 2017
17:3
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch02
DE: Method, Developments and Applications
39
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
// Initialization of population for j = 1 to Np for n = 1 to N Xj,n = rand(un, ln); // Selects a random value between upper and lower bounds of nth // variable in the search space specified end end // end of initialization do { for j = 1 to Np { r0 = rand (1:Np); // Select a random integer from 1 to Np for base vector r1 = rand (1:Np); // Select a random integer from 1 to Np for difference vector r2 = rand (1:Np); // Select a random integer from 1 to Np for difference vector // Note that r0, r1 and r2 must be distinct, and they also must be different from j. // Mutation and Crossover for n = 1 to N if rand(0,1) upper limit, then position = upper limit. If member position < lower limit, then position = lower limit. Evaluate fitness of the new member Selection
Output Finalize
Fig. 11.2 Flow chart of Differential Evolution used for the global optimization of xylitol production process (Ahlers et al., 1997)
To increase the diversity of the population, a crossover operation is used where the new solution vector ui,j is given by
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
Optimal Control of a Fermentation Process for Xylitol Production vi , j if rand ( j ) CR xi , j otherwise
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
ui , j
333
(11.23)
Here, rand(j) [0,1] is a random number and CR [0,1] is the crossover probability constant given by the user. The population is updated comparing the objective function values of ui,j and xi,j, where the solution vector that provides the best value for the optimization problem is retained (Storn and Price, 1997). Fig. 11.2 shows the flow chart of the DE algorithm used in this chapter. Note that a step was added to the original algorithm to avoid the generation of new members in infeasible regions of design variables due to the mutation and crossover. Parameters of this stochastic method were identified using preliminary calculations, and they are reported in Table 11.3. See Chapters 1 and 2 in this book for more details on DE and its developments. For interested readers, Chapter 4 describes two versions of DE with tabu list and FORTRAN programs for them while Chapter 5 presents MS Excel-based program for multi-objective DE with tabu list, parameter adaptation and termination criteria. Table 11.3 Parameters of DE, PSO and ABC used to optimize the xylitol production problem Algorithm DE PSO ABC
Parameter FDE CR c1 c2 w Nlimit
Suggested value 0.5 0.8 1.50 1.50 0.75 100
Additionally, two swarm-intelligence based algorithms, namely PSO and ABC, were used for comparison purposes. In PSO algorithm, the population members or potential solutions are designated as particles and the whole population as a swarm (Eberhard and Kennedy, 1995; Kennedy and Eberhard, 1995). Each of the particles is evaluated for the objective function. Each particle is associated with a velocity (ve) that is adjusted at each step towards both the best fitness that has achieved so far (xlocal) and the best value from all the swarm (xglobal). Equations 11.24 and 11.25 are used in PSO algorithm:
Differential Evolution in Chemical Engineering
334
9in x 6in
b2817-ch11
L. Koop et al.
vei+1, j wvei, j c1r1 xi,local xi, j c2 r2 xi,global xi, j j j
(11.24)
x i + 1 , j x i, j v e i+ 1 , j
(11.25) Start Generate initial swarm randomly
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Calculate initial velocity Evaluate fitness of each particle update global best Meet termination criteria
Yes
No Update local best Update velocity Update particle position
No
Evaluate if the particle is out of the feasible region
Replace vk + 1j = –0.5 vk+1j Yes If particle positon > upper limit, then position = upper limit. If particle positon < lower limit, then position = lower limit.
Output Finalize
Fig. 11.3 Flow chart of Particle Swarm Optimization used for the global optimization of xylitol production process (Kennedy and Eberhard, 1995; Schwaab et al., 2008; Eberhard and Kennedy, 1995)
Here, r1 and r2 are two random numbers, w is the inertia weight that was introduced by Shi and Eberhard (1999) to improve the PSO performance, while c1 and c2 represent the positive acceleration constants from the cognitive and social term of the Equation 11.24, respectively (Kennedy, 1997; Eberhard and Shi, 2011). If the value of c1 is too high relative to c2, particles can wander excessively and in an isolate way though the
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
Optimal Control of a Fermentation Process for Xylitol Production
335
search space. On the other hand, the reverse can make the particles rush prematurely toward a local minimum (Kennedy and Eberhard, 1995). For this study, values of PSO parameters were determined via preliminary calculations and selected values are reported in Table 11.3. Note that the velocity of each particle in the first/initial swarm was calculated using
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
vei, j 2 r3 1
yupp ylow 2
(11.26)
Here, r3 is a random number, yupp and ylow are the upper and lower limits for the optimization variables. Fig. 11.3 shows the flow chart of the PSO algorithm used in this study. The ABC algorithm was proposed by Karaboga (2005); it simulates the behavior of a honeybee swarm. The algorithm involves three different categories of bees, namely, employed, onlooker and scout bees. The first colony from ABC algorithm is formed with the same number of employed and onlooker bees (Karaboga and Basturk, 2007). Each of the employed bees is responsible for exploring a specific food source. Therefore, the number of employed bees indicates the number of solutions or food sources (Karaboga and Ozturk, 2011). Each employed bee makes a change in its first position using the following equation: vi, j xi, j i, j xi, j xk, j for i = 1,…, NP; j = 1, …, D
(11.27)
Here, Øi,j is a random number between [0,1] and k {1,2,…, NP} is calculated randomly but it must be different from i. If the new solution (vi,j) has a better fitness value than the older one, the bee memorizes the new position. On the other hand, if the new solution is worst, the bee keeps the first position in its memory (Karagoga and Akay, 2009). The employed bees are also responsible for sharing the obtained data with the onlooker bees, which choose a food source based on the provided information (Karagoga and Akay, 2009). The probability (Pi) for choosing a food source increases with the fitness value (nectar amount), which is calculated using (Karaboga and Basturk, 2007) Pi =
fiti NP n 1
fit n
(11.28)
Here, fiti represents the quality (fitness) of a given solution that is calculated with the next equation for minimization objective:
Differential Evolution in Chemical Engineering
336
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
fit i
9in x 6in
b2817-ch11
L. Koop et al. 1 1 fi
(11.29)
Here fi represents the value of the objective function. Similar to the employed bees, the onlookers perform a modification in their position and check the fitness of the new solution. The algorithm saves the position corresponding to the best result (fitness) (Karaboga and Ozturk, 2011). In ABC, when a food source cannot be improved through a pre-defined number of tests (Nlimit), the source is abandoned. In this case, the scout bees replace this abandoned source for a new one that is randomly selected (Karaboga and Ozturk, 2011). Fig. 11.4 presents the flow chart of the ABC algorithm and Table 11.3 contains the algorithm parameters used in this study. The number of population members (NP) = 105 was used for solving all examples by DE, PSO and ABC. Preliminary calculations indicated that this value offered a good tradeoff between efficiency and reliability of these optimizers for solving the examples in this study. The termination criterion for all the three algorithms was the same, namely, maximum number of iterations = 500. Finally, all examples were solved 10 times with random initial values for decision variables and the mean results were used for data analysis of each stochastic method. 11.4 Results and Discussion 11.4.1 Example 1: One-feed operation
First, the results obtained for a single feed stream containing the solution with both xylose and glucose are analyzed in this sub-section. The optimal feed rate and concentration profiles of biomass, substrates and product versus the process time for xylitol production using DE, PSO and ABC are shown in Fig. 11.5. Note that these profiles correspond to the process configuration where the operating time is fixed and the best run of each optimization method is reported. It is clear that DE and PSO algorithms showed similar optimal feed rate and concentration profiles in contrast to ABC, which obtained different process patterns for feed rate and xylitol concentration (Fig. 11.5). The statistical analysis of results obtained with each stochastic method is reported in Table 11.4. As stated, the reactor volume was handled as an equality constraint during the global optimization of this
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
Optimal Control of a Fermentation Process for Xylitol Production
337
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
fermentation process and problem formulation given by Equation 11.21 was used to satisfy this process restriction.
Fig. 11.4 Flow chart of Artificial Bee Colony used for the global optimization of xylitol production process (Karaboga and Basturk, 2007; Karaboga et al., 2014)
Differential Evolution in Chemical Engineering
338
9in x 6in
b2817-ch11
L. Koop et al.
Method
Table 11.4 Results of the optimization of xylitol production process using DE, ABC and PSO: Example 1 with one feed operation and fixed process time; E - extracellular
DE
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
PSO ABC GA*
Final concentration, g/L Substrate added, g Xylitol Production Final time, yield, rate, E. h Biomass Xylose Glucose xylitol Xylose Glucose g/g g/h 18.67 24.47 0.06 25.80 299.97 30.00 0.34 2.58 40.0 ±0.17 ±0.42 ±0.00 ±0.00 ±0.15 ±0.01 ±0.00 ±0.00 18.67 24.27 0.06 25.79 300.11 30.01 0.34 2.58 40.0 ±0.07 ±0.23 ±0.00 ±0.01 ±0.17 ±0.02 ±0.00 ±0.00 18.76 24.54 0.05 25.54 300.00 30.00 0.34 2.55 40.0 ±0.69 ±1.52 ±0.01 ±0.11 ±0.05 ±0.00 ±0.00 ±0.01 40.0 18.20 25.30 0.08 25.59 300.00 30.00 0.34** 2.56
*Results reported by Tochampa et al. (2015) using Genetic Algorithm (GA) **Yield calculated with data reported by Tochampa et al. (2015) and Equation 11.30.
For comparison, the results obtained by Tochampa et al. (2015) for the simulation of this process under the same operating conditions but using Genetic Algorithm (GA) are also reported in Table 11.4. Overall, DE and PSO showed the best performance for the global optimization of feed and concentration profiles of xylitol production. These methods were able to find slightly better production rate than GA. Herein, note that Tochampa et al. (2015) calculated the xylitol yield in a different way (i.e., by Equation 11.31 below). Specifically, two approaches can be used to calculate the xylitol yield, as follows: Xylitol yield
Xylitol yield
C xitV L t
(11.30)
f
M xyl add
CxitVL t
(11.31)
f
M xyl add CxylVL
tf
where Mxyl add is the amount of xylose added to the reactor. First approach considers that the xylitol produced (and released by the cells) is related to the total amount of xylose fed to the reactor, while second approach considers that the xylitol production is related to the xylose metabolized by the cells that is given by the denominator of Equation 11.31.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimal Control of a Fermentation Process for Xylitol Production
339
Fig. 11.5 Optimal feed rate and concentration profiles for the xylitol production using DE (a, b), PSO (c, d) and ABC (e, f): Example 1 with one feed operation fixed process time
Equation 11.30 was used for the results reported in this study where all the substrate feed in the reactor was considered in the calculation of the xylitol yield without subtracting any residual amount obtained at the end of the process. Note that the xylitol yield obtained from the best run of DE was equal to 0.51 using Equation 11.31, which is the same value reported by Tochampa et al. (2015) using GA algorithm. For illustration, the convergence behavior of population members of DE, PSO and ABC is reported in Fig. 11.6. These profiles show that DE is able to improve the value of objective function with less numerical effort than PSO and ABC. Note that PSO outperformed ABC, which required more functions evaluations (NFE) for improving the tested objective function. Overall, DE and PSO have better diversification and intensification strategies than ABC, and they can reach a high precision
Differential Evolution in Chemical Engineering
340
9in x 6in
b2817-ch11
L. Koop et al.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
in the solution obtained for this fermentation problem (as can be seen by much less scatter in Fig. 11.6).
Fig. 11.6 Convergence profiles of population members in the best runs of (a) DE, (b) PSO and (c) ABC in the global optimization of xylitol production process: Example 1 with one feed operation and fixed process time
Results for the optimization of xylitol production using operating time as a decision variable are reported in Table 11.5. Again, both DE and PSO showed the best performance for the global minimization of the objective function, and they found better xylitol yield and production rate than those obtained with ABC under this operating scenario. The best feed and concentration profiles for this process configuration are shown in Fig. 11.7. Process patterns in Fig. 11.7 show a clear trend in the fermentation system, where a higher feed rate appears in the beginning of the process operation. This trend is probably due to an initial requirement of a
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
341
maximum substrate amount for its conversion to the desired product. Note that this behavior cannot be permanent due to the inhibition of cells uptake caused by high substrate concentration in the culture medium. This substrate inhibition has been reported by Hunag et al. (2015); they analyzed the optimization of fed-batch fermentation using a mixture of substrates: glucose and fructose to produce ethanol, and reported similar feed rate profile (i.e., high feed rate in the beginning of the process). Experimental results with high feed rate in the beginning of the xylitol fermentation process have been obtained by Tochampa et al. (2015). The concentration profiles of biomass, substrates and product validate the model behavior showing that optimal xylitol productivities can be achieved with initial high substrate feed. Note that this trend is recommendable in short process times. In the case of large operating times, if this trend is applied, the reactor volume would be quickly filled affecting the continuous substrate supply. Consequently, the biomass would start to consume the produced xylitol to cell growth and maintenance. Details of the xylitol metabolism have been reported by Tochampa et al. (2005). From these results, it is clear that handling the operating time as an optimization variable implies a better production rate. However, the xylitol yield decreased in comparison to the simulations performed with the fixed process time (Tables 11.4 and 11.5). Based on the need to improve this variable also (i.e., optimized), this problem can be handled as a multi-objective optimization problem where the simultaneous optimization of both the productivity and xylitol yield for this fermentation process should be performed. Table 11.5 Results of the optimization of xylitol production process using DE, ABC and PSO: Example 1 with one feed operation and process time as a decision variable; E extracellular Method
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimal Control of a Fermentation Process for Xylitol Production
Final time, h
21.00 ±0.00 21.04 PSO ±0.19 19.70 ABC ±1.32 DE
Final concentration, g/L E. Biomass Xylose Glucose xylitol 16.14 37.78 0.07 21.60 ±0.05 ±0.12 ±0.00 ±0.00 16.18 37.66 0.07 21.64 ±0.11 ±0.50 ±0.00 ±0.20 15.47 41.66 0.06 19.63 ±1.30 ±4.45 ±0.02 ±1.47
Substrate added, g Xylitol yield, Xylose Glucose g/g 298.78 ±0.06 298.90 ±0.19 299.37 ±0.58
29.88 ±0.01 29.89 ±0.02 29.94 ±0.06
0.29 ±0.00 0.29 ±0.00 0.26 ±0.02
Production rate, g/h 4.11 ±0.00 4.11 ±0.00 3.98 ±0.05
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
342
9in x 6in
b2817-ch11
L. Koop et al.
Fig. 11.7 Optimal feed rate and concentration profiles for the xylitol production using DE (a, b), PSO (c, d) and ABC (e, f): Example 1 with one feed operation and using the process time as decision variable
11.4.2 Example 2: Two-feed operation Results of global optimization using DE, PSO and ABC for the Example 2 with two feeds and considering fixed fermentation time are summarized in Table 11.6. The xylitol yield and production rate found by DE and PSO were higher than those obtained by ABC. In general, this process configuration (i.e., two-feed operation) improves the performance of fermentation process, compared to single feed operation, giving a better production rate even with a fixed operating time. For example, comparing this process configuration with the example 1 using the operating time as a decision variable (which is the best configuration of the example 1 based on productivity analysis), DE was able to
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
Optimal Control of a Fermentation Process for Xylitol Production
343
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
increase both the xylitol yield and production in 24.1 and 6.3%, respectively. In particular, this example is useful to illustrate the importance of using a robust method for the global optimization of fermentation processes. Fig. 11.8 shows the best feed rate and concentration profiles obtained from the 10 runs performed with DE, PSO and ABC, and Fig. 11.9 shows the worst results for the tested algorithms.
Fig. 11.8 Best solution for the feed rate and concentration profiles in the optimization of the xylitol production using DE (a, b), PSO (c, d) and ABC (e,f): Example 2 with two feeds and fixed process time
For the case of DE, the best and worst solutions provided almost the same feed rate and concentration profiles. With respect to PSO, the best and worst glucose feed rate profiles showed some differences that consequently impacted the concentration profiles. However, the profiles obtained with ABC varied significantly. This finding indicates that DE
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
344
9in x 6in
b2817-ch11
L. Koop et al.
and PSO outperformed ABC in terms of solution quality and sensitivity with respect to the initial values of decision variables and random number sequences. Note that stochastic methods offer a probability (usually lower than 100%) for finding the global optimum of a problem at hand. This probability will depend on the type of stochastic optimizer and the difficulty of the optimization problem to be solved. Although ABC has proved to be a robust method for solving other global optimization problems (Karaboga and Basturk, 2008; Karaboga and Akay, 2009), results reported in this chapter indicate that it is not reliable for performing the global optimization of xylitol production process. Although this process configuration implies 10 more decision variables than Example 1, both DE and PSO were capable of performing reliably the optimization of feed rate and concentration profiles of this fermentation process. The fermentation model for xylitol production is also very sensitive to variations in the glucose feed rate, which can be concluded from the comparison of the best and worst profiles in Figs. 11.8 and 11.9. As stated, DE is a robust global optimizer where the best and worst solutions provide the same feed rate and concentration profiles. However, by comparing the feed rate profiles obtained with PSO or ABC, slight differences in xylose profile caused significant changes in the glucose feed rate profile for the first 4 h of process time. Consequently, the xylitol concentration profile changed in the beginning of the fermentation process. These process profiles proved that in the time interval where the glucose feed is high, there is almost no increase in the xylitol concentration. It occurs due to the xylose uptake inhibition effect caused by the glucose. Therefore, glucose should not be fed in large concentrations to improve the process performance. On the other hand, as already discussed in the Introduction, microorganisms can use an easily metabolized carbon source (e.g., glucose) to generate biomass and energy, so that most of the xylose can be converted to xylitol (Tochampa et al., 2005, 2015). This fact is also confirmed via the increment of the xylitol concentration in the culture medium after the high glucose consumption; see concentration profiles in Figs. 11.8 and 11.9. Note that there is a small but permanent glucose feed in all feed rate profiles, which indicates its relevance in the fed-batch operation. Therefore, even though glucose has an inhibition effect, its low concentrations may allow continuous cell growth.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimal Control of a Fermentation Process for Xylitol Production
345
Table 11.7 reports the results for this process configuration using the process time as a decision variable. Overall, all the three algorithms found a better production rate than those obtained with a fixed operating time in Table 11.6. DE and PSO showed an increment of 8.5% in xylitol productivity, while ABC improved this by 6.8%. With respect to the process time, this parameter was reduced by 34.3% using DE, 34.0% with PSO and 38.6% with ABC. This result shows that the control of the substrates in separate feed streams has a positive impact on the optimization of the fermentation process for productivity. However, xylitol yield decreased 16.7% using both DE and PSO and 20.0% with ABC (Table 11.7). These results again confirm that both xylitol productivity and yield are conflicting targets.
Fig. 11.9 Worst solution for the feed rate and concentration profiles in the optimization of the xylitol production using DE (a, b), PSO (c, d) and ABC (e, f): Example 2 with two feeds and fixed process time
Differential Evolution in Chemical Engineering
b2817-ch11
L. Koop et al.
For this last configuration with two feeds and variable process time also, DE and PSO outperformed ABC in terms of reliability for the optimization of xylitol production. For illustration, the best and worst solutions found by DE, PSO and ABC are reported in Figs. 11.10 and 11.11, respectively. Feed rate and concentration profiles of DE and PSO for the worse and best solutions are very similar while the substrate feed rate profiles found by ABC differ significantly causing a greater impact on the biomass concentration profile. These results confirm that ABC has the worst performance of tested stochastic methods and it is not reliable for this specific bioprocess.
Method
Table 11.6 Results of the optimization of xylitol production process using DE, ABC and PSO: Example 2 with two feeds and fixed process time; E - extracellular Final concentration, g/L Substrate added, g Final time, E. Biomass Xylose Glucose Xylose Glucose h Xylitol 19.12 21.79 0.14 21.87 240.54 59.73 DE 20.0 ±0.23 ±0.17 ±0.00 ±0.00 ±0.46 ±0.44 19.79 21.84 0.14 21.83 241.00 59.32 PSO 20.0 ±0.58 ±0.26 ±0.00 ±0.06 ±0.79 ±0.60 19.90 22.18 0.12 20.61 236.61 63.39 ABC 20.0 ±1.28 ±2.51 ±0.02 ±0.56 ±8.27 ±8.31
Xylitol yield, g/g 0.36 ±0.00 0.36 ±0.00 0.35 ±0.01
Production rate, g/h 4.37 ±0.00 4.37 ±0.01 4.12 ±0.11
Table 11.7 Results of the optimization of xylitol production process using DE, ABC and PSO: Example 2 with two feeds and using the process time as a decision variable; E extracellular Method
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
346
9in x 6in
Final concentration, g/L Substrate added, g Final Xylitol Production time, yield, rate, E. Biomass Xylose Glucose Xylose Glucose h g/g g/h Xylitol 13.13 19.31 24.92 0.11 15.56 205.05 94.92 0.30 4.74 DE ±0.07 ±0.12 ±0.27 ±0.01 ±0.08 ±1.18 ±1.16 ±0.00 ±0.00 13.20 19.35 24.90 0.11 15.63 205.21 94.09 0.30 4.74 PSO ±0.12 ±0.16 ±0.23 ±0.00 ±0.15 ±1.07 ±1.35 ±0.00 ±0.00 12.28 19.31 24.88 0.09 13.50 192.35 106.84 0.28 4.40 ABC ±1.22 ±1.41 ±3.26 ±0.02 ±1.38 ±11.61 ±11.73 ±0.02 ±0.09
As stated, the optimization of the xylitol production problem has been formulated using penalty function approach. Tested stochastic methods have different capabilities for handling infeasible solutions during global optimization. For illustration, Fig. 11.12 shows the penalty function value versus NFE during the optimization search for the last
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
Optimal Control of a Fermentation Process for Xylitol Production
347
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
configuration. It is clear that both DE and PSO offered the best performance for handling the constraint on the reactor volume. On the other hand, ABC generated several solutions outside the feasible region of the optimization problem. In fact, DE and PSO have better numerical performance to exploit and intensify the search, and they are more effective than ABC to escape from the infeasible region and to locate the global optimum.
Fig. 11.10 Best solution for the feed rate and concentration profiles in the optimization of the xylitol production using DE (a, b), PSO (c, d) and ABC (e, f): Example 2 with two feeds and using the process time as a decision variable
11.5 Conclusions This chapter illustrates the capabilities of DE, PSO and ABC for the global optimization of a fermentation process for xylitol production. Overall, results showed that DE algorithm is a reliable method for the
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
348
9in x 6in
b2817-ch11
L. Koop et al.
dynamic optimization of the xylitol production in a fed-batch fermentation process, and it can outperform both PSO and ABC algorithms. Regarding the fermentation process performance, highest productivity can be achieved by controlling the substrate feed streams separately, and the optimization of the process time may improve the xylitol productivity. However, xylitol productivity and yield are conflicting targets, which must be handled as a multi-objective optimization problem. Finally, results showed the effect of the control of the glucose addition to the culture medium. This substrate must be enough to provide energy for the microorganisms and cannot exceed a limit that generates a substrate inhibition effect.
Fig. 11.11 Worst solution for the feed rate and concentration profiles in the optimization of the xylitol production using DE (a, b), PSO (c, d) and ABC (e, f): Example 2 with two feed and using the process time as a decision variable
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimal Control of a Fermentation Process for Xylitol Production
349
Fig. 11.12 Penalty function values versus number of function evaluations (NFE) of the best runs of (a) ABC, (b) PSO and (c) DE in the global optimization of xylitol production process: Example 2 with two feeds and using the process time as a decision variable
Acknowledgments This work has been possible due to a scholarship from CAPES Foundation, Ministry of Education of Brazil, Brasília-DF 70.040-020, Brazil (CAPES scholarship student – Process number: BEX 6868/15-0). References Albuquerque T. L. de, Silva Jr. I. J. da, Macedo G. R. de, Rocha M. V. P. (2014). Biotechnological production of xylitol from lignocellulosic wastes: A review, Process Biochem., 49, pp. 1779–1789. Alford J. S. (2006). Bioprocess control: Advances and challenges, Comput. Chem. Eng., 30, pp. 1464–1475. Ashino R., Nagase M., Vaillancourt R. (2000). Behind and beyond the Matlab ODE suite, Comput. Math. Appl., 40, pp. 491–512.
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
350
9in x 6in
b2817-ch11
L. Koop et al.
Ahlers F. J., Carlo W. Di, Fleiner C., Godwin L., Keenan M., Nath R. D., Neumaier A., Phillips J. R., Price K., Storn R., Turney P., Wang F.-S., Zandt J. V., Geldon H., Gauden P. A., Brauer C., Shivaram K. R., Novikov D. (1997). Differential Evolution (DE) for Continuous Function Optimization (an algorithm by Kenneth Price and Rainer Storn). Available at http://www.icsi.berkeley.edu/~storn/code.html#matl [Accessed 15 February 2016]. Bond M., Dunning N. (2008). Sweeteners and Sugar Alternatives in Food Technology, ed. Mitchell H., Chapter 15 “Xylitol,” (Wiley). Caramihai M., Chirvase A., Fonteix C., Marc I., Fournier F., Misleanu R., Ungureanu C. (2007). Optimal fed-batch bioprocess control. An Advanced Approach, Comput. Aided Chem. Eng., 24, pp. 787–792. Carrasco E. F. and Banga J. R. (1997). Dynamic optimization of batch reactors using adaptive stochastic algorithms, Ind. Eng. Chem. Res., 36, pp. 2252–2261. Chang Y.-H., Chang K.-S., Hsu C.-L., Chuang L.-T., Chen C.-Y., Huang F.-Y., Jang H.D. (2013). A comparative study on batch and fed-batch cultures of oleaginous yeast Cryptococcus sp. in glucose-based media and corncob hydrolysate for microbial oil production, Fuel, 105, pp. 711–717. Chen 11., Jiang Z.-H., Chen S., Qin W. (2010). Microbial and bioconversion production of D-xylitol and its detection and application, Int. J. Biol. Sci., 6, pp. 834–844. Eberhart R. and Kennedy J. (1995). A new optimizer using particle swarm theory, Proc. Sixth International Symposium on Micro Machine and Human Science, pp.39–43. Eberhart R. C. and Shi Y. (2011). Computational Intelligence: Concepts to Implementations, Elsevier. Ferrari J. C., Nagatani G., Corazza F. C., Oliveira J. V., Corazza M. L. (2009). Application of stochastic algorithms for parameter estimation in the liquid–liquid phase equilibrium modeling, Fluid Phase Equilibr., 280, pp. 110–119. Hunag W.-H., Shieh G. S., Wang F.-S. (2012). Optimization of fed-batch fermentation using mixture of sugars to produce ethanol, J. Taiwan Inst. Chem. Eng., 43, pp. 1–8. Jin H., Chen 11., Yang J., Wu L., Wang L. (2014). Hybrid intelligent control of substrate feeding for industrial fed-batch chlortetracycline fermentation process, ISA T., 53, pp. 1822–1837. Kapadi M. D. and Gudi R. D. (2004). Optimal control of fed-batch fermentation involving multiple feeds using Differential Evolution, Process Biochem., 39, pp. 1709–1721. Karaboga D. (2005). An idea based on honeybee swarm for numerical optimization (Technical Report-TR06), Erciyes University. Karaboga D. and Akay B. (2009). A comparative study of artificial bee colony algorithm, Appl. Math. Comput., 214, pp. 108–132. Karaboga D. and Basturk B. (2007). A powerful and efficient algorithm for numerical function optimization: Artificial Bee Colony (ABC) algorithm, J. Global Optim., 39, pp. 459–471. Karaboga D. and Basturk B. (2008). On the performance of artificial bee colony (ABC) algorithm, Appl. Soft Comput., 8, pp.687–697. Karaboga D. and Ozturk C. (2011). A novel clustering approach: artificial bee colony (ABC) algorithm, Appl. Soft Comput., 11, pp. 652–657. Karaboga D., Gorkemli B., Ozturk C., Karaboga N. (2014). A comprehensive survey: artificial bee colony (ABC) algorithm and applications, Artif. Intell. Rev., 42, pp. 21–57.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch11
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimal Control of a Fermentation Process for Xylitol Production
351
Kennedy J. (1997). The particle swarm: social adaptation of knowledge, Proc. IEEE International Conference on Evolutionary Computation, pp.303–308. Kennedy J. and Eberhard R. (1995). Particle swarm optimization, Proc. IEEE International Conference on Neural Networks, 4, pp.1942–1948. Lee F.C., Rangaiah G.P. and Ray A.K. (2007). Multi-objective optimization of an industrial penicillin v bioreactor train using non-dominated sorting genetic algorithm, Biotechnology and Bioengineering, 98, pp. 586–598. Lim H. C. and Shin H. S. (2013). Fed-Batch Cultures: Principles and Applications of Semi-Batch Bioreactors, Cambridge University Press. Liu C., Gong Z., Shen B., Feng E. (2013). Modelling and optimal control for a fed-batch fermentation process, Appl. Math. Model., 37, pp. 695–706. MathWorks Inc. (2000). MATLAB 6.1, The MathWorks Inc. Peterson M.E. (2013). Xylitol, Top. Companion Anim. Med., 28, pp. 18–20. Reis L. dos, Fontana R. C., Delabona P. da S., Lima D. J. da S., Camassola M., Pradella J. G. da C., Dillon A. J. P. (2013). Increased production of cellulases and xylanases by Penicillium echinulatum S1M29 in batch and fed-batch culture, Bioresource Technol., 146, pp. 597–603. Rocha M., Mendes R., Rocha O., Rocha I., Ferreira E. C. (2014). Optimization of fed-batch fermentation processes with bio-inspired algorithms, Expert Syst. Appl., 41, pp. 2186–2195. Rómoli S., Serrano M. E., Ortiz O. A., Vega J. R., Scaglia G. J. E. (2015). Tracking control of concentration profiles in a fed-batch bioreactor using a linear algebra methodology, ISA T., 57, pp. 162–171. Roubos J. A., van Straten G., van Boxtel A. J. B. (1999). An evolutionary strategy for fed-batch bioreactor optimization; concepts and performance, J. Biotechnol., 67, pp. 173–187. Sarkar D. and Modak J. M. (2004). Optimization of fed-batch bioreactors using genetic algorithm: multiple control variables, Comput. Chem. Eng., 28, pp. 789–798. Storn R. and Price K. (1997). Differential Evolution — A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces, J. Global Optim., 11, pp. 341–359. Schwaab M., Biscaia Jr. E. C., Monteiro J. L., Pinto J. C. (2008). Nonlinear parameter estimation through particle swarm optimization, Chem. Eng. Sci., 63, pp. 1542–1552. Shi Y. and Eberhard R. C. (1999). Empirical study of particle swarm optimization, Proc. Congress on Evolutionary Computation, 3, pp.1945–1950. Sirisansaneeyakul S., Staniszewski M., Rizzi M. (1995). Screening of yeasts for production of xylitol from d-xylose, J. Ferment. Bioeng., 80, pp. 565–570. Tochampa W., Sirisansaneeyakul S., Vanichsriratana W., Srinophakun P., Bakker H. H. C., Chisti Y. (2005). A model of xylitol production by the yeast Candida Mogii, Bioproc. Biosyst. Eng., 28, pp 175–183. Tochampa W., Sirisansaneeyakul S., Vanichsriratana W., Srinophakun P., Bakker H. H. C., Wannawilai S., Chisti Y. (2015). Optimal control of feeding in fed-batch production of xylitol, Ind. Eng. Chem. Res., 54, pp. 1992–2000. Winkelhausen E. and Kuzmanova S. (1998). Microbial conversion of D-xylose to xylitol, J. Ferment. Bioeng., 86, pp. 1–14.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch12
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Chapter 12
Nested Differential Evolution for Mixed-Integer Bi-level Optimization for Genome-Scale Metabolic Networks Feng-Sheng Wang Department of Chemical Engineering National Chung Cheng University, 62102 Chiayi, Taiwan
[email protected] Abstract Numerous bi-level optimization methods have been used to determine optimal strain designs for the genome-scale metabolic networks of bacteria. Such bi-level optimization problems are generally reduced to single-level problems using strong duality theory. However, this approach can exponentially increase computation time because the number of decision variables is increased, and that a growth-coupled production strain would be obtained cannot be guaranteed. This chapter introduces an equality constraint to minimize and maximize flux variability in the strain design problem, which guarantees a growthcoupled strain. However, transforming a bi-level optimization problem into a single-level optimization problem by using the Kuhn–Tucker theory and an equality constraint is still generally difficult. A nested hybrid differential evolution algorithm is proposed that can easily solve the constrained optimization problem to obtain a set of growth-coupled production strains. It is tested through the simulation of the iAF1260 metabolic network of E. coli. 352
Differential Evolution in Chemical Engineering
9in x 6in
Optimization of Genome-Scale Metabolic Networks
b2817-ch12
353
Keywords: Bi-level Optimization, Hybrid Differential Evolution, Decision Making, Metabolic Engineering, Mixed-Integer Programming.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
12.1
Introduction
A bi-level optimization problem (BLOP) is a mathematical problem that involves two levels of optimization tasks (Mathieu et al., 1994; Bard, 1998; Yin, 2000; Angelo et al., 2013; Angelo and Barbosa, 2015; Website). Most engineering design and metabolic engineering problems involve a hierarchy of sub-problems in which an inner sub-problem must be solved on the basis of the settings of the outer-level sub-problems. Not only is such hierarchical treatment necessary for an accurate evaluation of a design or a process, it also ensures a systematic and computationally-tractable procedure. In such hierarchical problems, bilevel optimization is generally solved by reducing the problem to a single-level problem. A BLOP is a special type of multi-objective optimization (MOO) problem, and numerous algorithms have been proposed to solve such problems; these algorithms can be classified into three categories: vertex enumeration, Kuhn–Tucker and evolutionary algorithms (Bard, 1998; Website). Vertex enumeration algorithms cannot solve large-scale problems because they use brute-force computation to enumerate every integer variable. Kuhn–Tucker algorithms have been employed to reduce a BLOP to a single-level optimization problem by using strong duality theory. However, the computation time when such an approach is used can increase exponentially when the number of decision variables is increased. Bi-level optimization is frequently applied in the analysis of largescale complex systems such as strain design problems in metabolic engineering. Existing methods for solving such strain design problems cannot guarantee that a growth-coupled production strain will be obtained; therefore, post-processing is required to identify such strains. In general, such BLOPs can be considered a special category of MOO problems. Many methods for solving MOO problems have been proposed, each of which has advantages and disadvantages (Sawaragi et al., 1985; Rangaiah, 2009; Rangaiah and Bonilla-Petriciolet, 2013); these methods are of two types: generating and preference-based methods. Generating methods are used to yield the Pareto-optimal front of the MOO problem, and designers then use decision-making criteria to perform a tradeoff procedure and obtain the optimal result. Such methods
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
354
9in x 6in
b2817-ch12
F.-S. Wang
are referred to as a posteriori decision-making methods. Preferencebased methods are applied to solve problems for which designers have certain expectations of the optimal solution. In this scenario, a priori conditions can be implemented, and a tradeoff procedure can then be applied to obtain the final solution. However, neither of these methods necessarily solve all the problems associated with bi-level and multipleobjective decision-making. A few studies have considered solving BLOPs through evolutionary optimization, and most of the methods proposed are nested in nature, as discussed on the website of Evolutionary Bi-level Optimization (Website). One of the earliest evolutionary algorithms for solving BLOPs was proposed by Mathieu et al. (1994), who used a genetic algorithm at the outer level and linear programming at the inner level. Yin (2000) solved the outer-level sub-problem by using a genetic algorithm and the inner-level sub-problem by using the reduced gradient method. Differential evolution (DE) at both levels and nested DE with ant colony optimization have also been applied to solve BLOPs (Angelo et al., 2013; Angelo and Barbosa, 2015). However, such algorithms require lot of computations to determine an optimal solution for large-scale BLOPs, such as the rational strain design problem of genome-scale metabolic networks. This chapter introduces a nested hybrid DE (NHDE) algorithm to solve a strain design problem modeled using a genome-scale metabolic network. It is then applied to design an ethanol production strain, which was solved by the NHDE algorithm for the first time. The next section of this chapter presents a mixed-integer bi-level optimization problem, and then introduces the computational procedures of NHDE to solve such a problem. An example is used to illustrate the performance of the algorithm. Section 12.3 describes application to growth-coupled production strain design problem. The performance of NHDE is also illustrated in this section. Finally, conclusions of this study are provided in Section 12.4. 12.2
Nested Hybrid Differential Evolution (NHDE)
Consider a mixed-integer bi-level optimization problem in the following form:
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimization of Genome-Scale Metabolic Networks
Outer optimization problem: max F (x, z ) x,z subject to Inner optimization problem: f ( x, z ) max x subject to g ( x, z ) ≤ 0 h(x, z ) = 0 x min ≤ x ≤ x max G (x, z ) ≤ 0 H ( x, z ) = 0 z min ≤ z ≤ z max
b2817-ch12
355
(12.1)
Here, x is an n-dimensional vector of real variables; z is an mdimensional vector of integer variables; F(x, z) and f(x, z) are the outer/leader objective and inner/follower objective, respectively; and G(x, z), g(x, z), H(x, z), and h(x, z) are the vectors of constraints that must be satisfied as inequalities and equalities in the outer and inner optimization problems, respectively. The constraints represent the feasible space of the decision variables. The bi-level formulation in Equation 12.1 contains two objectives, and is therefore a special MOO problem. The inner optimization problem is a constraint of the outer optimization problem, the mathematical formulation of which can be also written as the alternative expression (see Equation 12.2). MOO problems usually have multiple optimal solutions, that is, Pareto-optimal solutions. However, the outer and inner problems can either cooperate or conflict with each other so that the BLOP does not necessarily have a Pareto-optimal solution. In general, it is not possible to directly use MOO algorithms to solve BLOPs. 12.2.1 Algorithm The hybrid DE (HDE) algorithm was extended from the original DE algorithm, which is used for stochastic optimization over the continuous space (Storn and Price, 1996; Storn and Price, 1997). The DE algorithm does not use binary encoding like simple genetic algorithms do
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
356
9in x 6in
b2817-ch12
F.-S. Wang
(Michalewicz, 1996), or a probability density function to self-adapt its parameters as an evolution strategy (Schwefel, 1995). Instead, DE and its variants (Chiou and Wang, 1999; Babu and Angira, 2006; Angira and Santosh, 2007; Chiou, 2007; Srinivas and Rangaiah, 2007) perform mutation on the basis of the distribution of the solutions in the current population. Thus, search direction and possible step sizes depend on the location of the selected individuals to calculate mutation values. The use of the mutation scheme in DE and its variants results in a faster convergence rate but also a higher probability of achieving a premature solution because the diversity of the population decreases while solving the problem. This drawback can be overcome by using a larger population size, but this approach increases the amount of computation time required to evaluate the fitness function. This is particularly relevant when DE is used to solve optimal control problems (Wang and Chiou, 1997). Chiou and Wang (1999) developed an HDE algorithm to overcome such drawbacks and have successfully solved several chemical and biological process optimization problems using the developed algorithm (Wang et al., 1998; Chiou and Wang, 1999; Wang, 2000; Wang et al., 2001; Chen and Wang, 2003; Tsai and Wang, 2005; Ko and Wang, 2006; Lin and Wang, 2007; Liu and Wang, 2008; Chen and Wang, 2010; Cheng and Wang, 2010). max F (x, z ) x,z subject to f ( x, z ) subject to x ∈ arg max g x z 0 ≤ ( , ) x h(x, z ) = 0 G (x, z ) ≤ 0 H ( x, z ) = 0 x min ≤ x ≤ x max z min ≤ z ≤ z max
(12.2)
Both DE and HDE were originally developed to solve nonlinear optimization problems, and not BLOPs. An NHDE algorithm, extended from HDE, was used to identify the integer variables of the optimization
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch12
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimization of Genome-Scale Metabolic Networks
357
problem described by Equation 12.2, and the inner optimization problem was then solved using a nonlinear/linear optimization solver. The basic operations of NHDE are similar to conventional DE algorithms, and a comparison between them is made in Table 12.1 and Fig. 12.1. DE and HDE algorithms are parallel direct search algorithms that use the Np vectors of the integer/real decision variables in optimization problems for the population of the generation G. They begin with a population of Np possible solutions selected assuming a uniform probability distribution for each decision parameter vector. The NHDE mutation operator, adopted from DE, is an essential component, which makes NHDE different when compared with other evolutionary algorithms. It uses the difference between two or four randomly chosen individuals as an evolutionary direction. The ith mutant individual ( zˆ G )i in generation G is obtained through the difference of two or four random individuals as expressed in the following form:
{
}
ˆ G )i INT (z G ) p + ρ G (z G ) j − (z G ) k + (z G )l − (z G ) m = , i 1,..., N p (z=
(12.3)
Here, random indices j , k , l , m ∈{1,…, N p } are mutually exclusive. The operator INT is used to round the real vector into an integer vector. In DE, the differential mutation factor ρG ∈ [0, 1.2] is fixed and set by the user to obtain faster convergence. ρG is used here to control the step length along the direction of search, and is a random number between zero and one generated by a uniform distribution generator, so that more diversified individuals are obtained. NHDE also includes an additional mutation strategy by applying a linear crossover of the ith individual and the best individual (zG)b to generate the parent individual, which is expressed as follows:
(= z G ) p ρ Gp (z G )b + (1 − ρ Gp )(z G −1 )i
(12.4)
Here, the factor ρpG is a random number between zero and one generated by a uniform distribution generator, and (zG-1)i indicates the ith mutant individual in the previous generation. The mutation operation may cause the mutant individual to escape the search domain (i.e., bounds are violated). If this occurs, it is replaced by a random number within the lower and upper bounds of the particular decision variable, thus restricting to the search domain. The choice of mutation factor for DE/NHDE is heuristic and random. When population diversity is low,
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch12
F.-S. Wang
358
candidate individuals rapidly cluster together such that the individuals cannot be further improved, and premature convergence occurs. Table 12.1. Basic operation of the DE and NHDE algorithms. CR = 0.5, ε = 0.05, Np = 20, and the maximum number of iterations = 200. ρG = random number between 0 and 1 Original DE
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
1. 2. 3. 4. 5.
1.
NHDE Representation and initialization
Representation and = z min , z max ), i 1,..., N p (z 0 )i uniformInt( initialization = Mutation Each individual is generated by an integer Crossover operation random number between zmin and zmax with Selection and evaluation uniform distribution Repeat steps 2 to 4 2. Mutation with rounding operation (zˆ G )i INT {(z G ) p + =
}
ρ G (z G ) j − (z G ) k + (z G )l − (z G ) m
3.
4.
5.
Crossover operation z Gji −1 , if a random number > CR z Gji = G j 1,..., n; i 1,..., N p = = zˆ ji , otherwise, Restriction operation G G min max z ji , z ji ∈ [ z j , z j ] z Gji = min max min max G uniformInt(z j , z j ), z ji ∉ [ z j , z j ] Selection and evaluation (a) For each individual, solve the inner optimization problem by a LP/NLP solver
(b)
f (x, z i ), s.t. g (x, z i ) ≤ 0, h(x, z i ) = 0, max x min max x ≤ x ≤ x
Compute solution
fitness
for
each
feasible
= fitnessi F (xi , z i ) + penalty
6.
Migration operation performed naturally or enforced if necessary = (z G )i uniformInt( = z min , z max ), if ζ ≤ ε [0,1] 7.
Repeat steps 2 to 6
Similar to conventional evolutionary algorithms, the local population diversity could be increased by using a crossover operation such as a binomial crossover. As previously described, DE and NHDE use the difference between two or four mutually independent individuals to determine the direction of search and obtain a mutant individual. This
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch12
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimization of Genome-Scale Metabolic Networks
359
differential mutation converges quickly so that most individuals cluster around the best candidate individual in some generations. Consequently, the population diversity and exploration capability diminish and clustered individuals are unable to reproduce more diversified individuals through the mutation operation because the weighted difference is nearly zero, as can be observed from Equation 12.3. The recombination of mutant individuals and their clustered parents further prevents the reproduction of a diversified population. Therefore, all individuals quickly cluster together and superior individuals cannot be generated through mutation and crossover operations. Create initial population to select individuals
Solve LP/NLP problem at 1st individual
Feasible ? No
Solve LP/NLP problem at 2nd individual
Yes
Feasible ?
Penalty = 0
No
...
Yes
Yes
Feasible ?
Penalty = 0
No
Penalty = 0
Compute penalty
Compute penalty
Compute penalty
Solve LP/NLP problem at Nth individual
Evaluate fitness and determine the best individual
Yes
New population of individuals
Migration criterion met? No
Mutation and crossover
No
Termination criterion met?
Yes
Optimal solution
Fig. 12.1 Flowchart of the NHDE algorithm for solving mixed-integer BLOPs
The migration operation of the NHDE algorithm is used to help individuals escape from the local cluster, but this operation is performed only if the population diversity falls below a desired level. The degree of population diversity ζ is introduced to check whether the migration operation should be performed. Each element of the ith individual (zG)i in
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch12
F.-S. Wang
360
generation G is referred to as a gene of the individual, and the gene diversity index dzji is given by
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
G z Gji z= = n; i 1,..., N p ; i ≠ b 1,..., 0, if= jb , j dz ji = 1, otherwise
(12.5)
Here, zjiG and zjbG are the jth gene of the ith and best individual at the Gth generation, respectively. dzji is set to zero if the jth gene of the ith individual is identical to the best gene; otherwise it is set to one (Chiou and Wang, 1999; Liao et al., 2001). ζ is defined as the ratio of total gene diversities to the total number of genes other than those of the best individual: Np
ζ =
n
∑ ∑ dz
i= 1,i ≠ b j = 1
ji
n( N p − 1)
(12.6)
Here, ζ has a value between zero and one. ζ = 0 implies that all the genes are identical to those of the best individual. ζ = 1, however, indicates that the current candidate individuals comprise a completely diversified population. The desired tolerance ε for population diversity is assigned by the user. ε = 0 corresponds to the migration operation being switched off, and ε = 1 corresponds to the performance of the migration operator at every generation. The user assigns ε ∈ [ 0,1] , and if ζ < ε, the NHDE algorithm performs migration operations to generate a new population and escape from a local point. If ζ > ε, the algorithm suspends the migration operation and maintains a constant search direction. The new individuals can be generated on the basis of the best individual z Gjb by means of non-uniformly random choice as (12.7). Its implementation is similar to the non-uniform mutation in genetic algorithms.
G z Gjb − z min j min G z jb + INT ρ1 ( z j − z jb ) , if ρ 2 < max G z j − z min j z ji = (12.7) G max G 1,..., m, i = 1,..., N p z jb + INT ρ1 ( z j − z jb ) , otherwise; j = One of advantages of evolutionary algorithms is that several flexible types of fitness can be used to evaluate survival individuals. The evaluation operation used in DE and NHDE consists of two selection
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimization of Genome-Scale Metabolic Networks
b2817-ch12
361
operators, the first of which is the selection of either a parent or its offspring, depending on which has higher fitness. If the fitness of the offspring is higher than that of its parent, the offspring is selected; otherwise, the parent is retained. The other operator selects the optimal individual in the population. Conventional evolution algorithms used the objective function in the optimization problem as a fitness function. However, the solution of BLOPs (which have hierarchical objectives) in this manner results in a turbulent optimization. A nested algorithm, such as NHDE, could overcome this drawback. The core procedure of the NHDE algorithm is the “selection and evaluation” operation as shown in the gray block of Fig. 12.1, which is where it differs from DE and HDE algorithms. The selection and evaluation operation for NHDE involves two additional steps. The evaluation step solves each linear or nonlinear programming (LP or NLP) problem that is posed by the inner optimization of each individual as shown in Fig. 12.1.
Inner optimization problem for the i th individual: max f (x, z i ) x subject to g ( x, z i ) ≤ 0 h(x, z ) = 0 i x min ≤ x ≤ x max
(12.8)
An optimal solution for each candidate individual is achieved when the LP or NLP problem is convergent, the set of which comprises a feasible solution to the BLOP. By contrast, the fitness of the outer problem is penalized, if it results in an infeasible solution. Thereafter, one-on-one fitness competition is used to select which trial individuals survive. 12.2.2 Example A test BLOP problem was used to evaluate the performance of the NHDE algorithm. The test problem is taken from Aiyoshi and Shimizu (1984) and Floudas et al. (1999). The problem involves minimizing the inner and outer objective functions as follows:
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
362
9in x 6in
b2817-ch12
F.-S. Wang Outer optimization problem: min F (x,= y ) min 2 x1 + 2 x2 − 3 y1 − 3 y2 − 60 x,y x,y subject to Inner optimization problem: 2 2 f= (y ) min ( y1 − x1 + 20 ) + ( y2 − x2 + 20 ) min y y subject to − x + 2 y1 ≤ −10 1 − x2 + 2 y2 ≤ −10 10 ≤ y1 , y2 ≤ 20 − x1 + x2 + y1 − 2 y2 ≤ 40 0 ≤ x1 , x2 ≤ 50
(12.9)
Aiyoshi and Shimizu (1984) report a local solution at x = (25, 30)T, y = (5, 10)T for which the outer objective function is 5 and the inner objective function is 0. Floudas et al. (1999) present another local solution at x = (0, 0)T, y = (–10, –10)T for which the outer objective function is 0 and the inner objective function is 200. The NHDE algorithm is implemented in the GAMS environment (See the Annex available under the Supplementary Material on the book website) and is applied to solve the problem on a 3.4 GHz Intel Core i7 CPU with 32 GB of RAM. The performance and solution quality of the NHDE algorithm depended on three setting factors: the tolerance ratio used in migration, population size and maximum number of iterations. The crossover factor CR and tolerance ratio were set to 0.5 and 0.05, respectively. A population size of 20 was used, and the maximum number of iterations was 50. The best solution obtained by NHDE was at x = (0, 30)T, y = (–10, 10)T for which the outer objective function is 0 and the inner objective function is 100, and was smaller than the reported solutions. The problem was solved 50 times using the NHDE algorithm, and the average computation time required to perform 50 iterations was about 2 min. Table 12.2 shows the best solution for solving progress. Forty eight out of 50 runs tried could achieve the optimal solution, but two runs obtained the local solution at x = (0, 0)T, y = (–10, –10)T.
Differential Evolution in Chemical Engineering
9in x 6in
Optimization of Genome-Scale Metabolic Networks
b2817-ch12
363
12.3 Application to Growth-Coupled Production Strain Design Problems
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
12.3.1 Problem description
The goal of metabolic engineering is to design optimal strategies to manipulate metabolic bacteria networks, maximizing the quantity of desired products (Orth et al., 2010; Palsson, 2015; Maranas and Zomorrodi, 2016). Various traditional strategies have been employed to achieve this goal, including the elimination of competing pathways, overexpression of genes to increase production pathway flux, and activation of inactive pathways to maximize the production rate of target compounds at the expense of the growth rate. Experiments using these strategies have observed that mutation always increases growth rate of cells and decreases production rate of desired products after adaptive evolution (Burgard et al., 2003). A promising and innovative method to solve this problem is the design of growth-coupled production strains that increase the production of target compounds as evolution occurs and growth rate increases. Model-based optimization is a crucial tool for the design of growth-coupled production strains (Lee et al., 2010; Orth et al., 2010; Palsson, 2015; Maranas and Zomorrodi, 2016). The constraint-based reconstruction and analysis approach focuses on the stoichiometric relationships of metabolic networks applied to the study of genome-scale metabolism. The strain design problem was formulated as a BLOP that consists of a bioengineering (outer) optimization problem and a cellular (inner) optimization problem. Numerous methods for solving this BLOP have been developed, including OptKnock (Burgard et al., 2003; Maranas and Zomorrodi, 2016), OptStrain (Pharkya et al., 2004), OptReg (Pharkya and Maranas, 2006), OptForce (Ranganathan et al., 2010), OptORF (Kim and Reed, 2010), EMILiO (Yang et al., 2011), and ReacKnock (Xu et al., 2013), which reduce the BLOP to a single-level mixed-integer linear programming problem using duality theory. However, the computation time when such an approach is used can increase exponentially when the problem
Differential Evolution in Chemical Engineering
364
9in x 6in
b2817-ch12
F.-S. Wang
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
dimension increases. Although the OptKnock algorithm required up to one week to predict a five-reaction knockout design using the E. coli iAF1260 model, it was the first constraint-based method used to predict strain designs for various substrates and products (Lun et al., 2009).
RobustKnock extended the OptKnock method to avoid non-uniquely growth-coupled strains through the definition of a max-min objective function. It attempted to yield a growth-coupled production strain by accounting for the presence of competing pathways in the smaller model iJR904 (Lun et al., 2009). However, RobustKnock applied the duality transformation to solve the inner optimization problems, and is thus still computationally burdensome. Evolutionary algorithms have been applied to identify the modulated genes of strain design problems using a userdefined objective function in which complicated nonlinear objective functions are used (Patil et al., 2005; Feist et al., 2010). However, it could not be guaranteed that a growth-coupled production strain would be obtained. A post-processing procedure is required to identify a growth-coupled strain. 12.3.2 Design formulation
This chapter introduces a one-stage decision-making problem to identify reaction deletions of growth-coupled production strains. The design of growth-coupled production strains tends to identify mutants that employ the smallest number of knockout enzymes to meet certain specifications (e.g., maximal product yield or maximal substrate-specific productivity) (Tepper and Shlomi, 2010), and can be formulated as a multi-objective decision-making problem (see Equation 12.10). In Equation 12.10, min FVA max FVA and argmax{vbioeng,…} = vbioeng denote the argmin {vbioeng,…} = vbioeng minimum objective value and maximum objective value of the corresponding flux variability analysis (FVA) problems, respectively. This inner optimization problem ensures that the lower bound of the production rate is equal to its upper bound so that a growth-coupled strain can be obtained. The next inner level, argmax{vcellular,…}, is a flux balance analysis (FBA) problem and is used to find the maximum max FBA of the knockout strain. Here, v is an ncellular growth rate vcellular dimensional vector of the intracellular fluxes, N is an m × n stoichiometric matrix (where m is the number of metabolites and n is the number of reactions), z is a vector of enzymes which are selected to be
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch12
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimization of Genome-Scale Metabolic Networks
365
knocked out, ΩKO is the set of knockout enzymes, and vzLB and vzUB are the lower and upper bounds of the reaction flux that do not belong to ΩKO. The flux is set to zero if the corresponding reaction is selected to be knocked out. The FBA and FVA problems are briefly explained in Appendix A. Detailed definitions are described in the articles (Orth et al., 2010; Palsson, 2015; Maranas and Zomorrodi, 2016). The two objective functions defined in Equation 12.10 correspond to the maximization of the respective bioengineering and cellular objectives. The first objective function is similar to that employed by RobustKnock (Tepper and Shlomi, 2010) for the obtainment of a growth-coupled production strain, but does not guarantee it, as mentioned previously. The introduction of min FVA max FVA ) in Equation 12.10 ensures that the the equality constraint ( vbioeng = vbioeng optimal designs are growth-coupled production strains with fixed target compound production rates.
min FVA max f1 = vbioeng z max FBA max f 2 = vcellular z subject to vbioeng vbioeng subject to subject to Nv 0= Nv 0 = = arg max arg min max FBA max FBA vcellular ≥ vcellular vcellular ≥ vcellular v v 0, z ∈ Ω KO 0, z ∈ Ω KO z z v= v= LB UB LB UB vz ≤ vz ≤ vz , z ∉ Ω KO vz ≤ vz ≤ vz , z ∉ Ω KO vcellular subject to arg max Nv = 0 v v= 0, z ∈ Ω KO z vzLB ≤ vz ≤ vzUB , z ∉ Ω KO
(12.10)
Table 12.2 The best solution for solving progress obtained by NHDE Iteration 1 5 15 20 38
min F 2.7973 0.1607 0.0313 0.0013 0
min f 87.4371 100.0258 100.001 100 100
x1 0.7419 0 0 0 0
x2 32.627 30.3214 30.0627 30.0026 30
y1 -10 -10 -10 -10 -10
y2 11.3135 10.1607 10.0313 10.0013 10
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch12
F.-S. Wang
366
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
12.3.3 Solving strategy Equation (12.10) is a bi-objective bi-level linear optimization problem (BOBLOP). The solution to the BOBLOP problem is similar to a multiobjective optimization problem, which includes a set of non-dominated Pareto-optimal solutions. Numerous methods have been devised to generate the Pareto-optimal front within multi-objective programming (Rangaiah, 2009; Rangaiah and Bonilla-Petriciolet, 2013). After a BOBLOP solution is obtained, an appropriate design must be selected from the Pareto-optimal front on the basis of what is desired. Wang and Wu (2013, 2015) proposed the fuzzy programming approach to convert BOBLOP to generate a criterion for making the decision. The decisionmaking maximization problem for two objectives is expressed as 2
max ∏ηi z∈Ψ
(12.11)
i =1
Here, the membership function ηi, corresponding to the satisfaction grade of the objective, is defined as
0, fi < fi LB LB fi − fi = ηi ( fi ) UB , fi LB ≤ fi ≤ fiUB LB fi − fi 1, fi > fiUB ; i = 1, 2
(12.12)
The set Ψ in Equation 12.11 comprises all feasible solutions that satisfy the equality constraint and FBA problem in Equation 12.10. The lower and upper bounds, fiLB and fiUB, for each objective depend on the preference of the user and are defined in advance. They can be respectively estimated by solving the FBA and FVA problems for the wild type strain. Having elicited the membership function for each objective function, the decision-making maximization is applied to determine the maximum satisfactory grade among the membership functions. Note that the problem is still a mixed-integer bi-level optimization problem, in which the set Ψ consists of the inner optimization problems as listed in Equation 12.10 that can be solved by the NHDE algorithm. The fuzzy decision algorithm controls the
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimization of Genome-Scale Metabolic Networks
b2817-ch12
367
magnitude of the association between various objectives when there is a tradeoff between them by using distinct aggregation functions. The iAF1260 metabolic model (Feist et al., 2007) of E. coli was used to evaluate the performance of the NHDE algorithm; it comprises 1668 metabolites and 2383 reactions. The bioengineering objective was to maximize the ethanol production rate, and the cellular objective was to maximize the biomass growth rate. The maximum glucose utilization rate was set to 20 mmol/(h·gDW), and the minimum cell growth rate and required ATP for non-growth-associated cell maintenance were set to 0.1 h-1 and 8.39 mmol/(h·gDW), respectively. To evaluate membership grades, fiLB and fiUB were obtained by minimizing and maximizing the flux through each reaction using flux balance analysis under glucose anaerobic conditions. All optimization problems were solved using the CPLEX solver accessed through GAMS on a 3.4 GHz Intel Core i7 CPU with 32 GB of RAM. The performance and solution quality of the NHDE algorithm depended on three settings: the tolerance ratio used in migration, population size and maximum number of iterations. The crossover factor CR and tolerance ratio ε were set to 0.5 and 0.05, respectively. A population size of 20 was used, and the maximum number of iterations was 200. min FVA max FVA , The strain design problem used the equality constraint, vbioeng = vbioeng to ensure that a growth-coupled strain was obtained, and the evaluation operation of the NHDE algorithm easily performed the equalityconstrained optimization problem in Equation 12.10. Fig. 12.2 shows the computational flowchart describing the solution of the inner optimization problem for each candidate individual. The FBA problem for the selected knockout enzymes was solved first to obtain the maximum biomass max FBA
min FVA max FVA and vbioeng growth rate vcellular , which was then used to obtain vbioeng through the minimization and maximization of the FVA problem, respectively. The growth-coupled constraint checked whether the optimal values were equal.
12.3.4 Results An optimal growth-coupled strain for ethanol production with enzyme knockout phosphate acetyltransferase (EC2.3.1.8) was predicted by the NHDE algorithm. The optimal biomass growth rate and ethanol production rate were calculated to be 0.442 h-1 and 31.854 mmol/(h· gDW), respectively. Numerous alternative feasible growth-coupled strain
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch12
F.-S. Wang
368
designs (Table 12.3) were also computed. For knockout enzyme acetate kinase A (EC2.7.2.1), ethanol production and biomass growth rate calculated were identical to those obtained with EC2.3.1.8. Therefore, the two strains are identical solutions.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Create initial population to select individuals
Same procedures as the left blocks for each zi
Solve argmax{vcellular,…} under the selected zi
Feasible solution?
No
Yes
New population of individuals generated by mutation and crossover of NHDE
Solve argmin{vbioeng,…} under the max FBA selected zi and vcellular
Feasible solution?
No
Yes Solve argmax{vbioeng,…} under the max FBA selected zi and vcellular
Feasible solution?
No
Yes min FVA max FVA = vbioeng vbioeng
No
Compute penalty
Yes Evaluate fitness and determine the best individual
Fig. 12.2 Computational flowchart describing the solution of the inner optimization problem for the zi candidate individual in the selection operation of the NHDE algorithm. Np blocks are parallelized to compute the corresponding fitness. The check point of each feasible solution ensures that the optimal solution of the inner optimization problem is found using an LP solver
2.3.1.8 (Phosphate acetyltransferase) 2.7.2.1 (Acetate kinase A) 2.7.2.15 (Propionate kinase) 2.3.1.54 (Formate C-acetyltransferase) 5.3.1.1 (Triose-phosphate isomerase) 1.2.1.10 (Acetaldehyde dehydrogenase) 5.3.1.9 (Glucose-6-phosphate isomerase) 1.4.1.4 (Glutamate dehydrogenase) 5.3.1.6 (Ribose-5-phosphate isomerase)
Acetyl-CoA + Phosphate Acetyl phosphate + CoA Acetate + ATP Acetyl phosphate + ADP ADP + Propanoyl phosphate ATP + Propionate
31.853
0.442
31.853
0.442
Acetyl-CoA + Formate CoA + Pyruvate
34.033
0.405
Dihydroxyacetone phosphate Glyceraldehyde 3-phosphate Acetaldehyde + CoA + NAD+ Acetyl-CoA + H+ + NADH D-Glucose 6-phosphate D-Fructose 6phosphate 2L-Glutamate + H2O + NADP Oxoglutarate + H+ + NADPH + Ammonium alpha-D-Ribose 5-phosphate D-Ribulose 5phosphate D-Allose 6-phosphate Allulose 6-phosphate
17.006
0.3743
16.285
0.4645
16.045
0.4946
15.988
0.5017
9.213
0.2980
(ackA, tdcD, purT) tdcD (pflB, pflD) tpiA adhE pgi gdhA (rpiA, rpiB)
369
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
pta
b2817-ch12
Regulated reaction
9in x 6in
EC number (Enzyme Name)
Optimization of Genome-Scale Metabolic Networks
Knock-out gene
Differential Evolution in Chemical Engineering
Table 12.3 Feasible knockout genes obtained by the NHDE algorithm, the corresponding encoded enzyme, and its regulated reaction.
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
370
9in x 6in
b2817-ch12
F.-S. Wang
It is known from biological databases such as KEGG and BRENDA that phosphate acetyltransferase (EC2.3.1.8) expression can be encoded by the gene pta. However, one of the three genes – ackA, tdcD or purT – can encode the expression of acetate kinase A (EC2.7.2.1). Moreover, tdcA also encodes the expression of propionate kinase (EC2.7.2.15), as listed in Table 12.3. Thus, expression of the knockout enzyme EC2.3.1.8 is inactivated by deleting the gene pta only. By contrast, to knockout either ackA or purT, the acetate kinase A was inactive, but both EC2.7.2.1 and EC2.7.2.15 were deleted if the gene tdcD was knocked out (Table 12.3). From Table 12.3, we observed that the ethanol production rate (34.033) was a little higher than the optimal result obtained from pta knocked out, but the biomass growth rate was lower if formate C-acetyltransferase (EC 2.3.1.54) was used as the knockout. This Pareto-optimal solution enhanced the production rate but sacrificed the optimal biomass growth rate. The other feasible solutions shown in Table 12.3 achieved only half the production rate compared with the Pareto-optimal solution. The flux envelope of each feasible solution was computed (Fig. 12.3) to explain why each design satisfied the equality constraint at the maximum biomass growth rate and was a growthcoupled production strain. However, the wild type strain is not the growth-coupled production. As mentioned previously, the strain design problem considers the additional equality constraint to ensure a growth-coupled strain is obtained. Few studies, however, have defined the inner optimization problems as constraints in the strain design problem. The equality constraint is generally difficult to implement, and the bi-level optimization problem in Equation 12.10 is not easily solved by OptKnock (Burgard, et al., 2003; Maranas and Zomorrodi, 2016). The OptKnock algorithm was implemented in the GAMS environment to solve the strain design problem excluding the inner optimization constraint. It could not find a converged solution within one week because of the huge number of deletion candidates in the genomescale iAF1260 model. By contrast, the NHDE algorithm obtained numerous growth-coupled strain solutions for ethanol production within the same period. To ensure that an optimal solution for each growth-coupled production strain design was obtained, the maximum number of iterations (200) were performed. The BOBLOP was solved 10 times using the NHDE algorithm, and the average computation time required to perform 200 iterations was about 15.75 min.
Differential Evolution in Chemical Engineering
9in x 6in
Optimization of Genome-Scale Metabolic Networks
b2817-ch12
371
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Moreover, Wang and Wu (2015) compared the performance of NHDE, OptKnock, and GLDS (Lun et al., 2009) at designing growthcoupled strains for succinate, D-lactate, formate and R-1, 2propanediol production. The NHDE algorithm could guarantee to obtain the growth-coupled production strains.
Fig. 12.3 Flux envelop of each growth-coupled strain design and wild type E. coli
12.4 Conclusions This chapter introduced a NHDE algorithm to solve mixed-integer bilevel optimization problems. A numerical example is used to illustrate the performance of the algorithm that the minimum solution can be obtained and it is smaller than the reported results. The NHDE algorithm is then applied to solve strain design problems in genome-scale metabolic networks. A strain design problem can be modeled as BOBLOPs, which are increasingly applied in the analysis of large-scale complex systems. Numerous algorithms have been developed to solve bilevel optimal gene-knockout problems, most of which transform the
Differential Evolution in Chemical Engineering
372
9in x 6in
b2817-ch12
F.-S. Wang
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
cellular optimization problem into a duality problem. Because of the complexity and dimensions of duality problems, applications of these methods are limited. Additionally, existing algorithms for bi-level optimal gene-knockout problems cannot guarantee the obtainment of a growth-coupled production strain, and thus need a post-processing procedure to identify them. The core procedure of the NHDE algorithm is the selection and evaluation operation that ensures each feasible solution is a growth-coupled production strain. Appendix A A.1 Flux balance analysis Flux balance analysis (FBA) is a constraint-based modeling approach in which the stoichiometry of the underlying biochemical network constrains the solution. The approach is often utilized by metabolic engineers to quantitatively simulate microbial metabolism. FBA is formulated as a linear optimization problem as:
max z = cT v v subject to Nv = 0 v LB ≤ v ≤ vUB v ∈ Rn
(A.1)
Here, the matrix N is an m × n stoichiometry matrix with m metabolites and n reactions and c is the vector representing the linear objective function. The decision variables v represent fluxes, and vectors vLB and vUB specify lower and upper bounds, respectively. FBA assumes that metabolic networks will reach a steady state constrained by the stoichiometry. The stoichiometric constraints lead to an underdetermined system; however, a bounded solution space of all feasible fluxes can be identified. The most widely used objective function in FBA of metabolic networks is the maximization of the cell growth rate built upon the assumption that the cell is striving to maximally allocate all available resource towards growth.
Differential Evolution in Chemical Engineering
9in x 6in
Optimization of Genome-Scale Metabolic Networks
b2817-ch12
373
A.2 Flux variability analysis
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Generally, linear optimization problems arising in FBA involve many optimal flux distributions leading to the same maximum cell growth rate. Flux variability analysis (FVA) is applied to determine the maximum and minimum values of all the fluxes that will satisfy the constraints and allow for the same optimal objective value. FVA is expressed as:
max/ min vi , i = 1,..., n v subject to Nv = 0 LB UB v ≤v≤v cT v ≥ z * v ∈ R n
(A.2)
Here, z* is a maximum solution to (A.1). Generally, we need to solve 2n optimization problems. However, the strain design problem (12.10) only requires to determine the maximum and minimum value of production rate of the desired product. The growth-coupled production strain is therefore defined as both values are identical. References Aiyoshi, E. and Shimizu, K. (1984) A solution method for the static constrained Stackelberg problem via penalty method, IEEE Trans. on Automatic Control, AC-29, 1111-1114. Angelo, J. and Barbosa, H. (2015) A study on the use of heuristics to solve a bi-level programming problem, International Transactions in Operational Research, 22, 861882. Angelo, J., Krempser, E. and Barbosa., H. (2013) Differential evolution for bi-level programming., 2013 IEEE Congress on Evolutionary Computation. IEEE, Cancun, pp. 470 - 477. Angira, R. and Santosh, A. (2007) Optimization of dynamic systems: A trigonometric differential evolution approach, Computers & Chemical Engineering, 31, 1055-1063. Babu, B.V. and Angira, R. (2006) Modified Differential Evolution (MDE) for Optimization of Nonlinear Chemical Processes, Computers & Chemical Engineering, 30, 989-1002. Bard, J.F. (1998) Practical Bi-level Optimization: Algorithms and Applications. Nonconvex Optimization and Its Applications. Springer US. Burgard, A.P., Pharkya, P. and Maranas, C.D. (2003) OptKnock: A bi-level programming framework for identifying gene knockout strategies for microbial strain optimization, Biotechnology and Bioengineering, 84, 647-657.
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
374
9in x 6in
b2817-ch12
F.-S. Wang
Chen, M.L. and Wang, F.S. (2010) Optimization of a fed-batch simultaneous saccharification and co-fermentation process from lignocellulose to ethanol, Ind. Eng. Chem. Res., 49, 5775-5785. Chen, Y.F. and Wang, F.S. (2003) Crisp and fuzzy optimization of a fedbatch fermentation for ethanol production, Ind. Eng. Chem. Res., 42, 6843-6850. Cheng, H.C. and Wang, F.S. (2010) Computer-aided biocompatible solvent design for an integrated extractive fermentation-separation process, Chem. Eng. J., 162, 809-820. Chiou, J.P. (2007) Variable scaling hybrid differential evolution for large-scale economic dispatch problems, Electr. Power Syst. Res., 77, 212-218. Chiou, J.P. and Wang, F.S. (1999) Hybrid method of evolutionary algorithms for static and dynamic optimization problems with application to a fed-batch fermentation process, Computers & Chemical Engineering, 23, 1277-1291. Feist, A.M. et al. (2007) A genome-scale metabolic reconstruction for Escherichia coli K12 MG1655 that accounts for 1260 ORFs and thermodynamic information, Molecular Systems Biology, 3. Feist, A.M. et al. (2010) Model-driven evaluation of the production potential for growthcoupled products of Escherichia coli, Metabolic Engineering, 12, 173-186. Floudas, C.A. et al. (1999) Handbook of Test Problems in Local and Global Optimization, p.223, Kluwer Academic. Kim, J. and Reed, J. (2010) OptORF: Optimal metabolic and regulatory perturbations for metabolic engineering of microbial strains, Bmc Systems Biology, 4. Ko, C.L. and Wang, F.S. (2006) Run-to-run fed-batch optimization for protein production using recombinant Escherichia coli, Biochem. Eng. J., 30, 279-285. Lee, F.C., Rangaiah, G.P. and Lee, D.Y. (2010) Modeling and optimization of a multiproduct biosynthesis factory for multiple objectives, Metabolic Engineering, 12, 251267. Liao, C.T., Tzeng, W.J. and Wang, F.S. (2001) Mixed-integer hybrid differential evolution for synthesis of chemical processes, Journal of the Chinese Institute of Chemical Engineers, 32, 491-502. Lin, H.T. and Wang, F.S. (2007) Optimal design of an integrated fermentation process for lactic acid production, AIChE J., 53, 449-459. Liu, P.K. and Wang, F.S. (2008) Inference of biochemical network models in S-system using multi-objective optimization approach, Bioinformatics, 24, 1085-1092. Lun, D.S. et al. (2009) Large-scale identification of genetic design strategies using local search, Molecular Systems Biology, 5. Maranas, C.D. and Zomorrodi, A.R. (2016) Optimization Methods in Metabolic Networks. Wiley. Mathieu, R., Pittard, L. and Anandalingam, G. (1994) Genetic algorithm based approach to bi-level linear programming, Operations Research, 28, 1-21. Michalewicz, Z. (1996) Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag Berlin Heidelberg, New York, U.S.A. . Orth, J.D., Thiele, I. and Palsson, B. (2010) What is flux balance analysis?, Nat Biotech, 28, 245-248. Palsson, B. (2015) Systems Biology: Constraint-based Reconstruction and Analysis. Cambridge University Press. Patil, K. et al. (2005) Evolutionary programming as a platform for in silico metabolic engineering, Bmc Bioinformatics, 6.
Differential Evolution in Chemical Engineering
9in x 6in
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Optimization of Genome-Scale Metabolic Networks
b2817-ch12
375
Pharkya, P., Burgard, A.P. and Maranas, C.D. (2004) OptStrain: A computational framework for redesign of microbial production systems, Genome Research, 14, 2367-2376. Pharkya, P. and Maranas, C.D. (2006) An optimization framework for identifying reaction activation/inhibition or elimination candidates for overproduction in microbial systems, Metabolic Engineering, 8, 1-13. Rangaiah, G.P. (editor) (2009) Multi-Objective Optimization: Techniques and Applications in Chemical Engineering, Vol. 1 in the Advances in Process Systems Engineering, World Scientific, Singapore. Rangaiah, G.P. and Bonilla-Petriciolet, A. (editors) (2013) Multi-Objective Optimization in Chemical Engineering: Developments and Applications. Wiley. Ranganathan, S., Suthers, P.F. and Maranas, C.D. (2010) OptForce: An optimization procedure for identifying all genetic manipulations leading to targeted overproductions, Plos Computational Biology, 6. Sawaragi, Y., Nakayama, H. and Tanino, T. (1985) Theory of Multi-objective Optimization. Orlando: Academic Press. Schwefel, H.P. (1995) Evolution and Optimal Seeking. John Wiley & Sons, Inc., New York, USA. Srinivas, M. and Rangaiah, G.P. (2007) Differential evolution with tabu list for global optimization and its application to phase equilibrium and parameter estimation problems, Ind. Eng. Chem. Res., 46, 3410-3421. Storn, R. and Price, K. (1996) Minimizing the real functions of the ICEC'96 contest by differential evolution. Evolutionary Computation, 1996., Proceedings of IEEE International Conference on. IEEE, Nagoya, pp. 842-844. Storn, R. and Price, K. (1997) Differential evolution — A simple and efficient heuristic for global optimization over continuous spaces, Journal of Global Optimization, 11, 341-359. Tepper, N. and Shlomi, T. (2010) Predicting metabolic engineering knockout strategies for chemical production: accounting for competing pathways, Bioinformatics, 26, 536-543. Tsai, K.Y. and Wang, F.S. (2005) Evolutionary optimization with data collocation for reverse engineering of biological networks, Bioinformatics, 21, 1180-1188. Wang, F.S. (2000) A modified collocation method for solving differential-algebraic equations, Appl. Math. Comput., 116, 257-278. Wang, F.S. and Chiou, J.P. (1997) Optimal control and optimal time location problems of differential-algebraic systems by differential evolution, Ind. Eng. Chem. Res., 36, 5348-5357. Wang, F.S., Jing, C.H. and Tsao, G.T. (1998) Fuzzy-decision-making problems of fuel ethanol production using a genetically engineered yeast, Ind. Eng. Chem. Res., 37, 3434-3443. Wang, F.S., Su, T.L. and Jang, H.J. (2001) Hybrid differential evolution for problems of kinetic parameter estimation and dynamic optimization of an ethanol fermentation process, Ind. Eng. Chem. Res., 40, 2876-2885. Wang, F.S. and Wu, W.H. (2013) Fuzzy Multi-Objective Optimization for Metabolic Reaction Networks by Mixed-Integer Hybrid Differential Evolution. In BonillaPetriciolet, G.P.R.a.A. (ed), Multi-Objective Optimization in Chemical Engineering: Development and Applications. Wiley
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
376
9in x 6in
b2817-ch12
F.-S. Wang
Wang, F.S. and Wu, W.H. (2015) Optimal design of growth-coupled production strains using nested hybrid differential evolution, Journal of the Taiwan Institute of Chemical Engineers, 54, 57-63. Website, Website for Evolutionary Bi-level Optimization http://www.bi-level.org/ (accessed in October 2016). Xu, Z. et al. (2013) ReacKnock: Identifying reaction deletion strategies for microbial strain optimization based on genome-scale metabolic network, Plos One, 8. Yang, L., Cluett, W.R. and Mahadevan, R. (2011) EMILiO: A fast algorithm for genomescale strain design, Metabolic Engineering, 13, 272-281. Yin, Y.F. (2000) Genetic-algorithms-based approach for bi-level programming models, Journal of Transportation Engineering-Asce, 126, 115-120.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Chapter 13
Applications of Differential Evolution in Polymerization Reaction Engineering Elena-Niculina Dragoi and Silvia Curteanu* Gheorghe Asachi Technical University of Iasi Faculty of Chemical Engineering and Environmental Protection Department of Chemical Engineering, Iasi, RO-700050, Romania *
Corresponding author:
[email protected]
Abstract This chapter focuses on applications of differential evolution (DE) in the field of polymerization reaction engineering. DE can be applied for modeling and for process optimization; it is being used in different variants, according to the specific characteristics of the system and the required accuracy. First, difficulties in modeling the polymerization processes are presented to justify the use of artificial intelligence tools, particularly artificial neural networks (ANN) and DE. These difficulties are related to the complexity of the reaction medium, lack of complete knowledge of the reaction mechanism, problems in developing and solving phenomenological models, their accuracy and/or potential for inclusion in on-line control procedures. Neuro-evolutive techniques are recommended methodologies for modeling and optimizing such complex polymerization processes. A special section is dedicated to general aspects of how DE can be used in combination with ANN for developing optimal neural models and for determining optimal operating conditions. Two applications: synthesis of polyacrylamide based hydrogels and of siloxane-siloxane copolymers, are discussed in detail.
377
Differential Evolution in Chemical Engineering
378
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
Keywords: Differential Evolution, Artificial Neural Networks, Neuroevolution, Modeling, Polymerization.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
13.1 Introduction Optimization, as means to find the best-suited solution for a given problem, is an important aspect influencing a variety of systems and processes. Considering that the majority of optimization problems are characterized by multimodal search spaces, have competing specifications and their parameters are limited by constraints, optimization is not an easy task. This requires efficient and powerful optimization approaches. The majority of classical optimization approaches are based on gradient methods, which, in case of complex systems, do not always generate near optimal solutions. The population based algorithms tend to provide good solutions for a majority of problems, which leads to their frequent use for solving different types of problems and systems ranging from synthetic problems (benchmarks) to real-life applications (Rangaiah, 2010; Precup et al., 2013; RamírezOrtegón et al., 2013; Kazakov and Lempert, 2015). From the group of population based algorithms, the evolutionary algorithms (EAs) are the oldest and better known. They have many advantages such as generality, reliability, robustness, do not require indepth knowledge about the problem being solved, and easy to understand and implement (Liu et al., 2010). In addition, EAs have the ability to escape the local minima and to provide near global minimum. In addition, they can be applied to problems having discontinuous, nondifferentiable and/or non-convex objective functions (Pant et al., 2009). Differential evolution (DE) is an EA that, compared with other algorithms from its class, is relatively simple and straightforward, and has good performance on a multitude of problems (e.g., Das and Suganthan, 2011; Sharma and Rangaiah, 2013). It was developed for solving difficult optimization problems on continuous spaces, and the literature indicates that it is the best when traditional approaches fail to provide acceptable solutions (Feoktistov, 2006). The main aspects that ensure DE success are self-adaptability, diversity control and continuous improvement (Feoktistov, 2006). These advantages make DE attractive for solving difficult and complex problems. Consequently, the areas of application for DE are large and vary substantially, including domains such as engineering, bio-informatics, environmental and climate control,
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Applications of DE in Polymerization Reaction Engineering
379
molecular biology. See Chapters 1 and 2 of this book, and also Dragoi and Curteanu (2016) for DE applications in Chemical Engineering. In this work, different DE versions in combination with artificial neural networks (ANNs) are applied to model a series of polymerization processes. The variants were developed to solve other chemical engineering systems, but, due to the DE flexibility, they can be applied to different processes, such as those belonging to the polymerization reactions. In order to assess the performance of the different DE strategies, two case studies were used; they are polyacrylamide based hydrogel synthesis and siloxane-siloxane copolymers synthesis. DE is a powerful algorithm but it was observed that, in some cases, it does not perform as expected (Thangaraj et al., 2009). The main situations in which DE encounters problems are related to difficult to explore search spaces and noisy environments (Neri and Tirronen, 2010). Also, DE suffers from premature convergence, stagnation, and is sensitive to the control parameters settings. In this context, different strategies were adopted to improve its performance. The modifications are in close correlation with a series of research directions: i) basic DE; ii) domain specific; iii) application specific, and iv) computing environment related (Storn, 2008). Another approach used for improvement is hybridization (i.e., mixing features or algorithms in order to outperform the parent or individual components). In case of DE, hybridization can be performed at different levels: i) individual; ii) population; iii) external and iv) meta-level (Feoktistov, 2006). See Chapters 4 and 5 of this book as well as Srinivas and Rangaiah (2007a, b) and Sharma and Rangaiah (2013) on hybridization of DE with tabu concept of tabu search. In this chapter, the three DE based algorithms used in combination with ANNs have multiple improvements belonging to different directions of research. 13.2 Difficulties in Modeling and Optimizing Polymerization Processes Generally, due to their specific features and to the general characteristics of the chemical processes, the polymerization reactions have a number of modeling and optimization related difficulties. They are complex and, often, their phenomenology is not fully elucidated. Also, elaborating credible phenomenological models involves precise knowledge of physical and chemical laws that govern the processes. Often, some
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
380
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
approximations are required, affecting the accuracy of model predictions. Moreover, complexities of the mathematical models cause extra difficulties in solving them and require a high computational time, which means the inability to use the models in on-line optimal control procedures. In these circumstances, empirical modeling based on inputoutput data becomes a preferable alternative to the phenomenological modeling, in terms of methodology and accuracy of model predictions. Additional modeling difficulties can be identified in the complex and diverse domain of polymerization reactions. Polymeric materials contain numerous products added to give them desirable properties. Since polymers with different properties are obtained by changing the initial conditions, the most commonly used method for the synthesis of these materials is batch polymerization. However, data collection for modeling is made difficult by the diversity of working conditions and properties, especially when adequate number and representativeness are necessary. Unlike discontinuous processes, continuous polymerizations require large amount of data, corresponding to a narrow operating domain. For homogeneous systems such as bulk and solution polymerizations, viscosity increases significantly as reaction progresses. This leads to difficulties in heat transfer associated with the exothermic nature of the reaction. In terms of kinetics, viscosity increase makes the initiation, propagation and termination to be controlled by diffusion. These are cage, glass and gel effects, which have to be included in the mathematical models. For this difficult to model part, only empirical relations can be successfully used. Another problem related to the polymerization reactors is due to the possibility of obtaining multiple answers. Various operating conditions (concentrations of reactants, temperature, pressure and catalysts) can lead to the same type of polymer — a polymer with the same molecular weight, density and composition — but with different yields. This highlights new challenges for mathematical modeling, which should make the distinction between these situations. Ensuring certain product quality is a much more difficult task for polymerization reactions than for reactions involving small molecules because molecular and morphological properties of the polymer strongly influence its physical, chemical, thermal, rheological and mechanical properties, and also the final applications of the polymer products. Hence, development of mathematical models to predict the polymer
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Applications of DE in Polymerization Reaction Engineering
381
product quality for given reaction conditions, is the key for efficient production and improvement in operating the plant (Tian et al., 2001). An alternative to the phenomenological modeling is the empirical one, which does not require a detailed knowledge about the process, is relatively easy to achieve because it relies on input-output data and, most often, provides reliable predictions. Statistical regression methods and ANNs are frequently used to develop empirical models. However, empirical models do not contribute to clarifying the reaction mechanism, and so they do not answer the question “why?”. Consequently, it would be useful to consider both types of models as complementary, in situations where this is possible. One type is mechanistic models that reproduce the phenomenology of the process despite less accurate results, and another type is empirical models, “black boxes” capable of providing better results (Curteanu, 2004). Combining the two types lead to hybrid models with better predictions. For example, a phenomenological model can be corrected with a neural network. Bhutani et al. (2006) described one application of such a hybrid model for an industrial hydrocracker optimization. Optimization and control of polymerization processes have significant impact on reducing the production costs and/or improving the quality of polymer products. Optimization of polymerization processes requires, above all, a mathematical model, thus taking over the difficulties associated with its development. Generally, it is a multi-objective problem because it involves several objectives simultaneously, often contradictory and incommensurable. Hence, solving such a problem is accompanied by difficulties, beginning with the formulation of the objective function and continuing with the selection of the working procedure and of the results of several possible options. For solving multi-objective problems, traditional methods based on differentiable functions or methods based on evolutionary algorithms can be used. Some specific disadvantages of the first category are: convergence to the optimal solution depends on the initial values, there is the danger of convergence to the local optimum, and the algorithms are dependent on the process to be optimized. These disadvantages can be overcome by using an EA, which generates a set of non-dominated solutions, known as Pareto-optimal front. Because of their flexibility, ease of operation, minimal requirements and global perspective, EAs have been successfully used in a wide variety of multi-objective problems. See Bhaskar et al. (2000), Rangaiah (2009), Rangaiah and
Differential Evolution in Chemical Engineering
382
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Bonilla-Petriciolet (2013) and Rangaiah et al. (2015) for applications of EAs to multi-objective optimization in chemical engineering. EAs do not need initial guesses and are likely to converge to the global optimum. In addition, they use information about the objective function only (and not its derivatives required by traditional optimization techniques), and they do not require any other auxiliary knowledge. 13.3 Differential Evolution and Neuro-evolution 13.3.1 Artificial neural networks ANNs are inspired by the biological brain and represent a powerful tool to model different relations between parameters. They are considered massive processors with a natural capacity to store experimental knowledge (Noor et al., 2010). An ANN is formed of a set of interconnected processing elements (nodes). An ANN node (also known as neuron) represents a mathematical simplification of a biological neuron; it generally has weighted inputs, sumator, activation function, bias and output. The weight of an input is a numerical value (positive or negative) indicating how strong the signal on that link is and influencing the output value. The sumator adds all the inputs and passes the result to the activation function to which, in order to generate an output, the bias is added. In contrast with the biological neuron, which has dynamic transfer functions driven by a complex internal structure, majority of the activation functions used for an ANN is represented by simple transfer functions (linear, sigmoid, Gaussian etc.) (Chandra and Yao, 2006). See Pirdashti et al. (2013) for a recent review covering applications of ANNs in chemical engineering. The determination of the optimal ANN parameters is known as training, and this process can be formulated as a minimization problem. This is a crucial aspect in using ANNs because it influences its performance. Training (or learning) is the manner in which the ANN acquires the knowledge about the system being modeled, and it is performed using example data (Yardimci, 2009). Three classes of learning are encountered. They are supervised (where a direct comparison between the ANN output and the desired one is performed), unsupervised (where no information about “correct answer” is available and the learning is performed based on only the correlation between input data), and reinforcement learning (where the exact desired output is
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Applications of DE in Polymerization Reaction Engineering
383
unknown and the training is performed based on the information of correctitude of the actual output) (Xin, 1999). The arrangement of the neurons and the nature of the connections determine the structure of an ANN. Since the development of perceptron (the first practical ANN), various types of networks were proposed: radial basis functions (RBF), self-organizing map (SOM), Hopfield neural networks and so on (Noor et al., 2010). Among all these types of networks, the most used type (and consequently the most known) is the multilayer perceptron (MLP), which is a fully coupled ANN having two or more layers in which each input activates all the hidden neurons. An example of a MLP with three layers (input, one hidden layer and output) is shown in Fig. 13.1. Here, m is the number of inputs, n is the number of neurons in the hidden layer, k is the number of outputs, b1…bn represent the biases of the hidden neurons, and bout1...boutk represent the biases of the output neurons. The most known training algorithm for MLP is the back propagation (BK), which is based on the gradient descent method. BK has the advantage of the directed search, the weights being updated with the only scope of minimizing the error (Kiranyaz et al., 2009). The main steps of the BK algorithm are: i) application of the input vector to the ANN in order to determine the output; ii) comparison of the actual output with the desired value and the determination of the error; iii) determination of the direction of change for each weight; iv) determination of the weight value; v) application of the correction to the weights; and vi) repeat the first five steps for all the input vectors until the error has an acceptable value (Freeman and Skapura, 1991). Despite its popularity, BK does not always provide acceptable results. It is depended on the learning rate parameter (which can introduce oscillations), can get trapped in local minima, and the computational resource required raises drastically when additional layers are added (Kiranyaz et al., 2009). In this context, researchers tried to find other approaches to train ANNs, and the use of neuro-evolutionary methods is a good alternative. 13.3.2 Neuro-evolution with differential evolution The combination of DE with ANNs belongs to the neuro-evolution area. When designing the link between the two, many aspects are considered. Neuro-evolution represents a special class of ANNs in which evolution is an additional form of adaptation and is a promising approach to solve
Differential Evolution in Chemical Engineering
384
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
reinforcement-learning problems (Xin, 1999; Kohl and Miikkulainen, 2009). Neuro-evolution encompasses different approaches, and the main elements used to distinguish them are: i) the ANN parameters being optimized and ii) the difference between the solutions represented as genomes (Islam and Yao, 2008).
Fig. 13.1 Generalized schema of a multi-layer perceptron (MLP)
Regarding the first aspect, evolution can be applied to: i) connection weights (training); ii) architecture (topology determination); and iii) learning rules (setting the optimal values for the training approach). The evolution of weights is the most used approach in neuro-evolution, and it was applied successfully from the beginning of this research domain (Jung and Reggia, 2008). The main problem of weight evolution (in the context of a fixed topology) is related to the issue of selecting the appropriate topology (Kohl and Miikkulainen, 2009). The evolution of architectures represents an automatic approach to ANN design, thus eliminating the human intervention and expertise through the application of the trial and error approach. Because the training performance is sensitive to the different architectures available, the evolution of learning rules (that can be seen as a “learning to learn” process) represents a natural way for the development of neuro-evolution approaches.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
385
The second aspect is related to representation (encoding). The best choice for the method of encoding depends on the characteristics of the problem being solved and on the available knowledge (Volna, 2010). The three main types are direct, indirect and implicit. In the direct encoding, there is a direct correspondence (one to one mapping) between the phenotype (ANNs) and the genotype (the encoded structure representing the ANN). The advantage of this encoding is that it can be easily decoded but its dimension is in close correlation with the network complexity (scalability problem). In the case of indirect encoding, the conversion of genotype to phenotype requires at least one level of interpretation (Boozarjomehry and Svrcek, 2001). The Lindenmayer systems represent the main source of inspiration for many indirect encoding strategies (Volna, 2010). The implicit representation derives from biological genetic regulatory networks, but it is scarcely applied in ANNs (Durr et al., 2006). Neuro-evolution is not only applied to simple ANNs but also to ensembles (Fig. 13.2). An ensemble (also known as stack) is a group of ANNs that work together to solve a specific problem. It adopts the “divide and conquer” approach and can perform more complex tasks than its components (Islam and Yao, 2008). The difference between evolving ANNs and evolving ensembles of ANNs is that, in the first case, the solution is represented by an ANN, while, in the second case, the solution is a population of ANNs (Liu, 2006). Stack inputs
Stack inputs
Stack inputs
. . .
. . .
Fig. 13.2 Example of a stack of ANNs
. . .
Stack output
. . .
. . .
. . .
. . .
. . .
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Applications of DE in Polymerization Reaction Engineering
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
386
Table 13.1 Recent examples of DE-ANN studies DE variant
Application
Kişi (2013)
Training
Not specified
Architecture and learning params.
Fully connected MLP
DE/rand/1/bin in association with Time series forecasting BK for learning
Training
Feedforward ANN
Orthogonal DE with LevenbergMarquardt algorithm
Training
Wavelet ANN
Improved DE (IDEA) where F and CR are adjusted using PSO
Training
BK ANN
Adaptive DE in association with Time series forecasting BK for learning
Training
Multilayer ANN architecture
Not specified
Pan evaporation for monitoring, survey and management of water resources.
Donate et al. (2013)
UCI machine learning repository (medical datasets)
Zhang et al. (2014)
DE in combination with Pseudoinverse for learning
Liao (2014)
RBF
ANN
Wang et al. (2014a)
Learning params.
Use
Oyebode and Adeyemo (2014)
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Dash et al. (2013)
Ref.
Not specified
3 function approximation and classification (breast cancer, diabetes, heart)
Air-conditioning load forecasting
Water management
(Continued )
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
387
Two-layer ANN Not specified MLP, Product-Unit ANN, AdaptiveNetwork-Based Fuzzy Inference Systems, Wavelet ANN Not specified Convolution ANN
Training Partial architecture and training
Ensemble of DE strategies (nnDE) with modified trial vector generation. The strategies are randomly selected from Best/1/ exp, Rand /1/exp, Rand -tobest/1/exp, Best/2/ exp, Rand/2/ exp, Best/ 1/bin, Rand/1/bin, Rand-to-best/1/bin, Best/2/ bin, Rand/2/bin
Training
DEGL (Das et al., 2009), ADE (Yu et al., 2014), AM-DEGL (Piotrowski, 2013), ATPS-DE (Zhu et al., 2013), CoBiDE (Wang et al., 2014b), DE-DPS Streamwater (Sarker et al., 2014), EG-Cr-DE (Guo and Yang, 2015), MGBBDE temperature prediction (Wang et al., 2013), Rcr-JADE (Gong et al., 2014), SapsDE (Wang and Zhao, 2013), DEGLRcr-EG-JADE (merging elements of DE variants)
Training
Training
Du et al. (2015) Piotrowski et al. (2016)
Non-dominated sorting adaptive DE (NSJADE) with DE/currentto-pbest strategy
Fister et al. (2016)
Self-adaptive DE with strategies Ultra-short-term power DE/ran1/1, DE/Best/1, DE/randload forecasting to-best/1, DE/best/2 , DE/rand/2
Rasdi et al. (2016)
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Liu et al. 2014)
Table 13.1 (Continued )
Short-term replenishment forecasting problem in fashion industry
CEC 2014 Problems
Not specified
MNIST and CIFAR datasets
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
388
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
In summary, neuro-evolution refers to the optimization of parameters including architecture of simple and ensemble ANNs for better model predictions. It does not refer to optimizing the process system for better operating conditions, for example, to maximize profit. The improved model from neuro-evolution can later be used for optimizing the system. Owing to its good performance, in simple or in modified versions, DE was applied effectively to evolve ANNs. Some recent examples are given in Table 13.1, where the column “Ref.” gives the reference. 13.3.3 DE variants and their application Depending on the characteristics of the algorithm, there are a multitude of DE approaches (see Chapters 1 and 2 of this book for an introduction to DE and its principles, applicable to general optimization problems). In this chapter, three variants are tested on a series of polymerization processes. These variants are represented by SADE-NN-1 (Dragoi et al., 2013b), SADE-NN-2 (Dragoi et al., 2012) and hSADE-NN (Curteanu et al., 2014b). At their core, these algorithms combine different versions of DE with ANNs in order to model and to optimize specific aspects of chemical engineering problems. They were designed to perform simultaneous optimization of ANN parameters and topology; after that, the determined model(s) can be used to optimize the studied problem/system. These algorithms represent incremental improvements and, consequently, there are a series of common aspects such as ANN encoding, fitness function, data workflow, self-adaptive mechanism, mutation principle, stop criteria and normalization. Encoding For the considered variants, the ANN encoding used is direct encoding. Each of the ANN parameters selected for optimization is represented by a real value in the individuals of the DE population. Since a simultaneous parametric and structural optimization of the ANN is performed, the number of parameters included in the individuals is large and refer to: i) number of hidden layers (Nhl), ii) number of neurons in each hidden layer, iii) weights (Wi), iv) biases (Bi), v) activation functions (Ai) and vi) parameters of activation functions (Pi). Considering the fact that the bigger the network the bigger is the encoded structure, the maximum number of hidden layers is set to two. This value is chosen based on theoretical and practical considerations, as it was observed that a network
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
389
with two hidden layers and a sufficient number of neurons in the hidden layers could model almost any process with an acceptable accuracy. The structure of the genotype is presented in Fig. 13.3. Nhl
Nh1
Nh2
W1
.. Wm B1
.. Bn
A1
.. An
P1
.. Pn
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Fig. 13.3 The encoded neural network
Taking into consideration that there are x inputs and y outputs, for a network having Nhl = 2, the number of parameters (Nparams) is given by: Nparams=1+2+(x*Nh1+Nh1*Nh2+Nh2*y)+3*(x+Nh1+Nh2+y) Nhl
Nh1 and Nh2
weights
(13.1)
bias and activation function
The above equation simplifies to: Nparams=3+x*(Nh1+3)+Nh1*(Nh2+3)+Nh2*(y+3)+3*y
(13.2)
Depending on the characteristics of the problem being solved, Nparams varies taking into account not only the limit on the number of hidden layers but also the limits on the number of neurons in each hidden layer. These limits are problem specific and are imposed to control the computational time. On the other hand, once the maximum length of the genotype for a specific problem is set, the length of each individual remains unchanged and the position of each parameter is fixed during the evolution. Fitness function For the determination of optimal ANNs, two main steps are performed: training and testing. From the available dataset, a portion of the data is employed for each step and performance indexes are generated based on these two groups of data. In order to assess the performance of each model, a fitness function was used. For all the DE variants used in this chapter, the fitness is inversely proportional to the mean squared error (MSE) in the training phase. The scope of keeping the testing data outside the evolution (training phase) is to ensure that the generalization capabilities of the model determined are correctly evaluated and do not influence the training process in any manner.
Differential Evolution in Chemical Engineering
390
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
Workflow of data
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
In order to model and/or to optimize a problem, a series of steps must be performed. The experimental data is collected, analyzed, and used to generate the model and to optimize the system (Fig. 13.4).
Fig. 13.4 Steps for modelling and optimizing a system; the black box represents the model
At each generation, new individuals representing encoded networks were created and each ANN (which is determined after decoding) calculates the fitness function. Then, the DE algorithm uses the fitness in order to determine the best individuals for the next generation. An example of influence of the process data on the steps of DE is presented in Fig. 13.5 by bold arrows, where “1” indicates that the process characteristics influence the ANN structure and “2” indicates the use of the experimental data for fitness computation (Dragoi et al., 2016). Fig. 13.5 exemplifies the most complex DE version used in this work, where two algorithms BackPropagation (a known ANN training procedure) and Random Search (an optimization approach that does not use derivatives) are randomly selected at each generation to improve the local best solution. The fitness function is computed after crossover and for the application of the local optimizers.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Applications of DE in Polymerization Reaction Engineering
391
Fig. 13.5 hSADE-NN algorithm with data workflow [from Dragoi et al. (2016)]
Mutation principle The role of the mutation principle in the DE algorithm (as in the majority of EAs) is to introduce perturbation in the population in order to maintain diversity (Thangraj et al., 2010). One of the methods to improve the DE performance is to propose new mutation strategies. Consequently, for the considered variants, a modified mutation approach, based on existing mutation approaches but with the individuals ordered using their fitness function, is used. The manner in which the individuals are selected and
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
392
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
applied in the mutation remains unchanged. For example, for the ‘Rand/1’ version, three random vectors are selected (x1, x2 and x3). After that, they are arranged based on the fitness function. Suppose that the final order is x2, x3, x1, then the mutated individual will be xm=x2+F*(x3x1), where F is the mutation factor. Consequently, the base vector is the one being the fittest among the selected individuals. In other words, the mutation becomes an operation in which the best individual from a group is changed in order to create a better individual. In case of “Best/1” and “Best/2” versions, the base vector is selected as the best individual in the population, and the rest are randomly identified. In this way, the randomly identified individuals (that participate in the differential term(s)) are arranged based on their fitness. This is a greedy approach, based on the idea that better individuals can create better children. Self-adaptive mechanism Taking into account the fact that finding the optimal control parameters is not an easy task (as their values are problem dependent), the use of alternative procedures relieves the user from manually setting them. There are four main types of tuning the control parameters (Feoktistov, 2006): i) deterministic (a deterministic law is used to evolve the control parameters and there is no feedback from the search); ii) adaptive (there is feedback from the search procedure); iii) self-adaptation (the control parameters are introduced into the algorithm itself); and iv) hybrid. In this context, the DE based variants used in this chapter have a selfadaptive principle, which evolves the control parameter simultaneously with the other parameters of the individuals. The main idea of the selfadaptive variant used consists of including F and Cr control parameters into the individual and evolving them using the same equations as the other parameters. In this manner, the complexity of adding selfadaptation does not differ from the complexity of non-adaptive variants, and no other operation is required (such as applying other relations to the control parameters or saving them in separate structures). Stopping criteria The general stopping criterion used for majority of EAs including DE in the literature is the maximum number of generations (usually set at the beginning of the algorithm). See Zielinski and Laur (2008) for other stopping criteria, classified into improvement based, movement based
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
393
and distribution based. The variants used in this chapter employ a stopping criterion that combines two conditions: MSE for the training data is smaller than 10–8 and the maximum number of generations. The algorithm will stop when one of these two conditions is satisfied.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Normalization Normalization is one of the most common tools used for improving the performance of automated recognition systems (Priddy and Keller, 2005). Normalization of input data for ANNSs has several advantages such as reducing the estimation error and the training computational time (Leeghim et al., 2008). There are different normalization procedures that can be used. For the current DE-based ANN determination approaches, a variant called “min-max normalization” is employed because it constrains the range of each input feature (Priddy and Keller, 2005). It is described by: 𝑥𝑥𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 = 𝑚𝑚𝑚𝑚𝑚𝑚𝑡𝑡 + (𝑚𝑚𝑚𝑚𝑚𝑚𝑡𝑡 − 𝑚𝑚𝑚𝑚𝑚𝑚𝑡𝑡 )
𝑥𝑥−𝑚𝑚𝑚𝑚𝑚𝑚
𝑚𝑚𝑚𝑚𝑥𝑥−𝑚𝑚𝑚𝑚𝑚𝑚
(13.3)
Here, xnorm represents the result, mint = – 0.9 and maxt = 0.9 are the values determining the target interval, and min and max describe the interval of x values. Particularities The elements mentioned above (encoding, fitness, data workflow, mutation, self-adaptive mechanism and stopping criteria) are common to all the three DE variants used in this chapter for modeling polymerization processes. However, each variant has its specific characteristics that distinguishes it from the other. For example, SADE-NN-1 has an initialization based on opposition based learning (OBL). SADE-NN-2 uses BackPropagation as a local search procedure. On the other hand, hSADE-NN uses OBL at the initialization phase and a local search procedure where Back-Propagation and Random Search algorithm are randomly applied to improve performance. Consequently, hSADE-NN is a combination of SADE-NN-1, SADE-NN-2 and Random Search. SADE-NN-1 This variant was proposed by Dragoi et al. (2013b), where it was applied to model and to optimize an aerobic fermentation process when using two different broths represented by Propionibacterium shemanii
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
394
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
(bacteria) and Saccharomyces cervisiae (yeast). The scope was to identify the influence of n-dodecane on the oxygen transfer, which was modeled as a function of biomass concentration, superficial air velocity, specific power and oxygen volumetric fraction. In order to determine the influence of the OBL principle on the DE performance for the two broths, four simulation cases were considered, combining initialization (classical and OBL), mutation (classical and modified) and six DE base variants (Rand/1/Exp, Rand/1/Bin, Best/1/Exp, Best/1/Bin, Rand-toBest/1/Exp, Rand-to-Best/1/Bin). The simulations results for bacteria based broths indicated that the best ANN model was determined using OBL initialization and modified mutation with Best/1/Bin strategy. In the case of yeast fermentation, the best model was determined using OBL initialization and normal mutation with Best/1/Bin strategy. After the best models were determined, sensitivity analysis was performed to determine the influence of each input on the oxygen transfer, the results obtained were in concordance with the experimental data. In the final step, using the best models, the fermentation process was optimized to determine the experimental conditions for maximizing oxygen transfer. SADE-NN-2 SADE-NN-2 was proposed by Dragoi et al. (2012), where it was applied to monitor and model a pharmaceutical freeze-drying process. Since freeze-drying is a complex process with many parameters, different combinations of inputs were first considered. In addition, a recurrence relation of the model was tested, and the final configuration of the model has 9 inputs and 2 outputs. The inputs were time (t), chamber pressure (Pc), fluid temperature (Tfluid), product temperature (delayed one, two and three times) and dried layer length (delayed one, two and three times), and the outputs were product temperature (Ti) and dried layer length (Ldried). The delay considered is based on the data frequency collection, which was 600 s because the dynamics of the process was slow. As the process is complex and the involved parameters are difficult to experimentally measure, the ANN was designed based on data generated by a mono-dimensional phenomenological model using heat transfer coefficients determined in another study. The scope was to generate a simpler model with a similar performance that can then be used in systems with low resources (e.g., for control purposes). In order to determine the best model, two DE variants were used, Best/1/Bin and Rand-to-Best/1/Bin, and the former had higher performance. After a
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Applications of DE in Polymerization Reaction Engineering
395
series of tests related to the performance and prediction capacity of the model, the ANN was validated on a set of experimental data (Dragoi et al., 2013a). The tests were performed in two cases (modeling and monitoring) and also included data with different variations of heat transfer coefficient and resistance of the dried layer and with fixed or variable Pc and Tfluid conditions. In case of modeling, given the system state, the model predicts the state at the next point (defined by the frequency of data collection). In case of monitoring, based on the state of the system at a specific time, the ANN generates the dynamics of the entire system and predicts the drying time and Ti. Another process for which SADE-NN-2 approach was used is the corrosion resistance of TiMo alloys under different conditions (Mareci et al., 2015). The objective was to model the polarization resistance based on immersion time, caffeine concentration, NaF concentration, Ti content and pH. In the previous case study (freeze drying), the DE based methodology was used in combination with simple MLP. For this case study, the ANNs were simple or organized in stacks. The motivation for using stacks was that simple models were not able to capture the process dynamics and to generate acceptable predictions. In addition, it was observed that there are significant differences in the behavior when having low (3-4) and high (5-8) pH and, consequently, two different models were determined, one for each case. SADE-NN-2 was also applied for predicting the partition coefficient of guanidine hydrochloride in poly (ethylene glycol) 4000/ phosphate/ guanidine hydrochloride/water system (Pirdashti et al., 2015), and for modeling the barrier properties of high-density polyethylene/polyamide 6/clay nanocomposites (Moghri and Dragoi, 2015). hSADE-NN hSADE-NN was proposed in Curteanu et al. (2014b), and was applied to model the depollution process of gaseous streams containing n-hexane. Three different adsorbents were studied for adsorption of n-hexane vapors from gaseous stream, in the dynamic regime. The experimental study was carried out by varying some important parameters that influence the efficiency of the adsorption process like concentration of volatile organic compounds in the gaseous stream, temperature and length of the adsorbent bed in the adsorption column. To assess the performance of the modifications in hSADE-NN in comparison with SADE-NN-2, the two methods were used to determine models for the
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
396
0.062 0.061 0.06 0.059 0.058 0.057 0.056
0.7 0.6 0.5 MSE value
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
considered process. The best network determined with hSADE-NN has the highest fitness (value of 16.478) and the topology 5:18:1. Fig. 13.6 shows the training process and the stopping point according to low MSE in training and testing phases. A comparison with a descriptive method (Yoon and Nelson, 1984) indicated that the DE-based methodologies are suitable to find good models with acceptable accuracy.
0.4
0
0.3 0.2
MSE training
0.1
50
100
MSE testing
0 0
200
400 600 Generations
800
1000
Fig. 13.6 Evolution of MSEtraining and MSEtesting during the determination of neural network model using hSADE-NN algorithm [from Curteanu et al. (2014b)]
13.4 DE Applications in Polymerization Engineering This section reviews some applications of DE in the polymerization field. Process optimization requires a model of the system. Different types of models exist. The most used, especially in the chemical engineering area, are phenomenological models, based on the physical and chemical laws governing the process. Due to the difficulty of solving complex mathematical relations or poorly known interactions, phenomenological models are not always viable, the ANNs coming as good alternatives to them. Consequently, in this section, the focus is on two main aspects: DE in combination with phenomenological models and DE with ANNs as models for the polymerization processes. In both
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
397
cases, there are only a few studies; this indicates that DE is scarcely used in the area of polymerization reactions although it shows great potential.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
13.4.1 DE and phenomenological models One of the first case study to apply DE for optimizing a polymerization process is Lee et al. (1999). The problem solved is the dynamic optimization of a continuous methyl methacrylate-vinyl acetate copolymerization reactor. The outputs were the weight-average molecular weight and the composition of vinyl acetate in the dead copolymer, while the inputs were the jacket temperature and the monomer flow rate. The objective function is: 𝑡𝑡𝑡𝑡
Min 𝐽𝐽 = ∫𝑡𝑡𝑡𝑡 (𝜔𝜔1 |𝑀𝑀𝑤𝑤 (𝑡𝑡) − 𝑀𝑀𝑤𝑤𝑤𝑤 | + 𝜔𝜔2 |𝑌𝑌𝑉𝑉𝑉𝑉 (𝑡𝑡) − 𝑌𝑌𝑉𝑉𝑉𝑉𝑉𝑉 |)𝑑𝑑𝑑𝑑 (13.4)
𝑢𝑢1,𝑢𝑢2
Here, u1 is the jacket temperature (K), u2 is the feed flow rate of monomer (kg/h), t0 is the initial time, tf is the final time, Mw is the molecular weight, Mwd is the desired molecular weight, YVA is the composition of vinyl acetate in the dead copolymer, YVAd is the desired composition of vinyl acetate in dead copolymer, ω1 = 1.0 × 10–7 and ω2 = 0.01. Lee et al. (1999) optimized the process using the classic DE and a modified DE, which included improvement with local search and search space reduction through the introduction of heuristic constraints. The modified DE was better at finding the optimal solution. The addition of NaOH to the epoxy polymerization influences the process significantly but no consistent rules are known on this aspect, which leads to a suboptimal operation of the reactor (Raha et al., 2004). DE and genetic algorithm were applied to determine the optimal NaOH addition. The model used for this optimization is a detailed kinetic scheme, where various monomer and polymer species were considered. The problem can be formulated as a standard nonlinear programing problem, and two specific cases were optimized by Raha et al. (2004). In case 1, the molecular weight is maximized through the control of hourly NaOH addition. In case 2, the scope was to minimize the polidispersity index by controlling the NaOH addition pattern. 13.4.2 DE and ANN Neuro-evolutionary techniques, in particular ANNs combined with DE, are useful for chemical engineering processes by realizing two main actions: getting neuronal network model in optimal form and,
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
398
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
subsequently, optimizing the process. Fig. 13.7 presents such an ANNDE methodology. In Curteanu et al. (2010), a polymerization process was modeled by ANNs developed with DE and also with other methods. The considered process was the free radical polymerization of styrene accompanied by gel and glass effects, and the scope was to model the monomer conversion and molecular masses, as a function of reaction conditions. With this goal, three variables are chosen as inputs of the neural models: initiator concentration, I0, temperature, T, and reaction time, t, and the three outputs of the networks are monomer conversion, x, numerical average molecular weight, Mn, and gravimetrical average molecular weight, Mw. A simple version of DE was applied with a fitness function combining MSE in the training and testing phases; the neural network parameters (number of hidden layers, number of neurons in these layers, weights between layers, biases of the neurons and activation functions) were optimized. Different values of DE parameters Cr, F, and the number of epochs were tried because the algorithm is very sensitive to their values. The best model having the maximum fitness value (obtained when Cr = 0.99, F = 0.7 and number of epochs = 5000), has 9 neurons in the first hidden layer and 7 neurons in the second hidden layer; its performance is considered acceptable for the selected process. The same process of free radical polymerization of styrene was modeled with ANNs and support vector machines (SVM), developed with DE and clonal selection (CS) algorithm (Curteanu et al., 2014a). In the DE-SVM procedure, SVM represents the model of the process and DE performs parameter optimization of the model. In the case of DEANN combination, the population was formed of neural models. In the case of DE-SVM, the individuals in the population are lists of SVM parameters such as SVM type (µ-SVR and ε-SVR, where SVR is support vector regression), kernel type (linear, polynomial, RBF and sigmoid), degree (applicable only for polynomial kernel type), γ (coefficient of polynomial and sigmoid kernels) and C, the cost parameter. Fig. 13.8 presents some example results of applying all the four methods. The accuracy of the results and the efficiency of the applied methods depend on the specific dynamics of the free radical polymerization (where the diffusion effects appear) and on the initiation conditions (reaction temperature and initiator concentration). Consequently, at different working conditions, a different method could be suitable. For instance,
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
399
when temperature is 383 K and initial value of initiator is 20 mol/l, the best approach is DE-SVM.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Process parameters
Model Artificial Neural Networks
Process
Differential Evolution
Fig. 13.7 ANN-DE methodology for optimizing the model and the process
In order to model the conversion of vinyl chloride monomer in the polyvinylchloride polymerization process, a RBF neural network optimized by a cultural DE was applied (Wang and Guo, 2013). The traditional control approach for this process has monomer conversion lower than 85%. Since there are a multitude of factors (molecular weight, porosity, absorption rate, vinyl chloride residue, and thermal stability) that can influence this aspect, a soft model that can predict the conversion rate and conversion velocity can be a useful tool for improving performance. The process variables considered as inputs for the RBF model were: temperature inside kettle, pressure inside kettle, water flow rate of baffle, water flow of clip set, water feed flow rate, cooling water inlet temperature, water outlet temperature in clip set, water outlet temperature of damper and outlet temperature of cold water tank. Using the kernel principal component analysis (KPCA) approach, the model inputs were reduced. After that, the cultural DE was applied to optimize the centers and widths of the hidden layer base functions and the weights; the final number of parameters for optimization was 75. The
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
400
results obtained indicated that the approach has a high estimation accuracy, thus being suitable for a model predictive control system. 67000 0.81 62000
0.71
57000 Mn
0.51 x
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
0.61 0.41
52000
0.31 0.21
47000
0.11 0.01 0
10000
20000
30000
42000 1000
Time (s) Expected CS-NN DE-SVM
SADE-NN-2 SVR
11000 Time (s)
Expected CS-NN DE-SVM
SADE-NN-2 SVR
Fig. 13.8 Comparison between the predictions obtained with the four methods and the expected data for monomer conversion when temperature is 368 K and initial value of initiator is 10 mol/l (left plot), and numerical average molecular weight when temperature is 383 K and initial value of initiator is 20 mol/l (right plot)
Although apparently simple, the polycondensation reaction leading to polyazomethine is difficult to control due to its equilibrium character, the conversion degree being influenced by a series of parameters. The reaction between a siloxane diamine, 1,3-bis(3-aminopropyl) tetramethyldisiloxane, and terephthalaldehyde was performed in solution (in tetrahydrofuran) without by-products removal and in the absence of any catalyst or pH modifier (Macsim et al., 2015). Different conditions (co-monomers ratio, dilution and temperature), considered as input parameters for the process modeling were varied according to a preestablished experimental program. hSADE-NN was then applied for developing ANN models. A recurrent neural network (Fig. 13.9) provided very good results with the simulation data in agreement with the experimental data.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Concentration
Temperature
Neuron 1
Neuron 2
AP0:AT
Hidden layer
...
Vascozity
Time
Neuron 19
401
t-1
Neuron 20
Recurrent connection
Fig. 13.9 Structure of the obtained ANN model [from Macsim et al. (2015)]
13.5 Case Studies In order to model the two case studies (described below), the same settings were used for the three DE based approaches. The DE variant employed is RandToBest/2/Bin with a population size of 200 and 500 generations. For the network structure in this work, one hidden layer with maximum 30 neurons was considered. For each case study, two situations were considered: i) a single model is determined for all the selected outputs, and ii) a separate model is determined for each output. The purpose of this is to show that, in some cases, considering each output of the process as a separate problem, the model performance can be improved and thus the predictions can be more useful. In order to compare the two situations, the same datasets, with the same exemplars and in the same order, were used. Also, for all the case studies, the same approach was followed for data pre-processing. First, data is randomized (in order for the networks not to learn from a specific area of the dataset) and then it is split into two sets: 75% for training and 25% for testing.
Differential Evolution in Chemical Engineering
402
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
13.5.1 Polyacrylamide based hydrogels synthesis The variation of the main parameters in the synthesis of polyacrylamide based hydrogels was previously modeled to correlate with reaction conditions using a direct neural modeling approach (Curteanu et al., 2008). These hydrogels have multiple applications: gel electrophoresis, soft contact lenses, thickener and suspending agents, subdermal filler for aesthetic facial surgery, soil conditioner on farmland, construction sites for erosion control or such materials for the removal of total organic content in water treatment plants; therefore, good models are useful to predict specific properties. Polyacrylamide gels are usually obtained through copolymerization of acrylamide with a bifunctional monomer or by radical polymerization of acrylamide followed by a crosslinking reaction. For this complex process, “single step” method for polymerization/crosslinking is applied (Curteanu et al., 2009). Also, a biodegradable interpenetrated polymer, forming semi- and interpenetrated multicomponent networks based on polyacrylamide was added. In order to model this process, seven input variables were considered: CM (monomer concentration), CI (initiator concentration), CA (crosslinking agent concentration), PI (amount of inclusion polymer), T (temperature), t (reaction time) and type of included polymer codified as 1 – no polymer added, 2 – starch, 3 – polyvinyl alcohol (PVA) and 4 – gelatin. The outputs of the neural model were η (yield in crosslinked polymer) and α (swelling degree). After the experimental data was pre-processed, a series of 100 simulations (modeling runs) were performed for each case of DE variant and outputs. Multiple runs were performed due to the stochastic nature of the DE algorithm, the scope being to determine multiple good solutions for the considered process. Table 13.2 presents the best, worst and average results out of 100 runs for each case. In this table, MSE is mean squared error, while Corel Train 1, 2 and Corel Test 1, 2 correspond to the correlation in the training and testing phases for outputs one and two. Output one is the yield and output two is the swelling degree. In order to indicate the topology (in column “Top.”), a notation in the form of ‘inputs: neurons_hidden: outputs’ is used. For example, 7:07:01 indicates a network having 7 inputs, 7 neurons in the hidden layer and one output. In Table 13.2, B, W and A indicate respectively the best, worst and average results. The “Fit.” column presents the fitness, MSEtr the MSE in the training phase, MSEte the MSE in the testing phase, CTr 1, 2 the correlation in the training phase of the first and second output, and CTe
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
403
1, 2 the correlation in the testing phase of the first and second output. In some cases, some information is not available and this is indicated in Table 13.2 with “-”. For example, when models for each process output process were determined, the number of outputs of the model is one. Table 13.2(a) Results obtained with SADE-NN-1
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Out. 1
2
All
Fit.
MSEtr
MSEte
CTr 1
CTr 2
CTe 1
CTe 2
Top.
B
14505
7E-05
4E-04
0.984
-
0.975
-
7:07:01
W
4520
2E-04
1E-03
0.949
-
0.913
-
7:07:01
A
7848
1E-04
7E-04
0.969
-
0.950
-
-
B
1840
5E-04
2E-03
0.897
-
0.851
-
7:08:01
W
641
2E-03
3E-03
0.691
-
0.845
-
7:19:01
A
1154
9E-04
2E-03
0.817
-
0.855
-
-
B
2533
4E-04
2E-03
0.967
0.878
0.941
0.859
7:19:02
W
1325
8E-04
3E-03
0.877
0.802
0.807
0.813
7:06:02
A
18457
6E-04
2E-03
0.940
0.834
0.913
0.836
-
CTe 2 0.843 0.809 0.834
Top. 7:18:01 7:08:01 7:05:01 7:19:01 7:11:02 7:07:02 -
Table 13.2(b) Results obtained with SADE-NN-2 Out. 1
2
All
B W A B W A B W A
Fit. 14168 5181 7970 1654 719 1137 2217 1461 1823
MSEtr 7E-05 2E-04 1E-04 6E-04 1E-03 9E-04 5E-04 7E-04 6E-04
MSEte 4E-04 8E-04 7E-04 2E-03 3E-03 3E-03 2E-03 2E-03 2E-03
CTr 1 0.984 0.956 0.970 0.885 0.708 0.815 0.962 0.923 0.943
CTr 2 0.859 0.793 0.831
CTe 1 0.973 0.939 0.952 0.861 0.856 0.854 0.947 0.893 0.916
As can be observed in Table 13.2, in case of each DE based variant, the results obtained with the individual networks are slightly better than the model combining the two outputs (in terms of MSE and correlation). For output 1 (yield), the best fitness is obtained with SADE-NN-1, although the average results generated by this variant are the worst. This is also observed in the case of swelling degree output.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
404
Table 13.2(c) Results obtained with hSADE-NN Out. 1
All
MSEtr
MSEte
CTr 1
CTr 2
CTe 1
CTe 2
Top.
10158
1E-04
5E-04
0.978
-
0.961
-
7:12:01
W
9495
1E-04
5E-04
0.976
-
0.962
-
7:16:01
A
8119
1E-04
7E-04
0.971
-
0.952
-
-
B
1647
6E-04
2E-03
0.885
-
0.855
-
7:15:01
W
698
1E-03
3E-03
0.701
-
0.855
-
7:17:01
A
1222
9E-04
2E-03
0.830
-
0.853
-
-
B
2397
4E-04
2E-03
0.959
0.874
0.942
0.860
7:15:02
W
1445
7E-04
2E-03
0.927
0.785
0.887
0.817
7:03:02
A
1865
5E-04
2E-03
0.941
0.837
0.914
0.834
-
100 90 80 70 Yield
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
2
Fit. B
60 50 40 30 20 10 0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19
Exemplars Experimental
SADE-NN-1
SADE-NN-2
hSADE-NN
Fig. 13.10 Comparison between experimental data and predictions generated by the three DE based variants for the yield output
A point by point comparison was also performed in case of the test data for the two outputs (Fig. 13.10 for yield and Fig. 13.11 for swelling degree). In case of yield, the majority of predictions are in the close proximity of the experimental data but, for swelling degree, the
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
405
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
differences are quite high. These differences are also indicated by the MSE and correlation values presented in Table 13.2.
Compared to the previous similar approaches (Curteanu et al. 2008, 2009; Leon et al. 2010), the results presented here are new and show the advantages of the ANN-DE techniques, which leads to optimal neural networks. Better results are obtained with different DE methodologies (SADE-NN-1, SADE-NN-2 and hSADE-NN), especially for swelling degree. These results indicate DE capabilities for efficiently modeling complex relations. 8000 7000 6000 5000 4000 3000 2000 1000 0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 Exemplars
Experimental
SADE-NN-1
SADE-NN-2
hSADE-NN
Fig. 13.11 Comparison between experimental data and predictions generated by the three DE based variants for the swelling degree output
13.5.2 Siloxane-siloxane copolymers synthesis Polysiloxanes (also named silicones) are the most important inorganic polymers, the representative term being polydimethylsiloxane. Silicone polymers containing organic groups other than methyl or specific organic function on the chain or at its ends have opened new fields of applications, which are a result of siloxane chemical reactivity, solubility, miscibility, lubricity, etc.
Differential Evolution in Chemical Engineering
b2817-ch13
E.-N. Dragoi and S. Curteanu
406
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
9in x 6in
In this case study, ANNs were developed for modeling the dependence between the monomer conversion and copolymer composition and working conditions. The experimental data (80 exemplars) was gathered by ring-opening copolymerization of the octamethylcyclotetrasiloxane (D4) with 1,3,5,7-tetravinyl-1,3,5,7tetramethylcyclotetra-siloxane (D4V), with a cation exchange (styrenedivinylbenzene copolymer containing sulfonic groups) as a catalyst, in the absence of solvent (Curteanu and Cazacu, 2007). Consequently, the network has four inputs (t - reaction time, T - temperature of the copolymerization process, C – amount of catalyst and f1 – initial composition of the reaction mixture) and two outputs (x - reaction conversion and F1 – copolymer composition). As in the polyacrylamide based hydrogels synthesis case study, 100 modeling runs were performed with each DE variant, and the general statistics obtained are presented in Table 13.3. The notation used is that described for Table 13.2. Comparing the correlation in the training and testing phases between the models determined for output 2 (F1 parameter) and those for both the outputs, it can be observed that the best values obtained are close to each other. A point-by-point comparison between these two cases (i.e., output 2 alone and both outputs) for the SADE-NN-1 and hSADE-NN variants is presented in Fig. 13.12. Table 13.3(a) Results obtained with SADE-NN-1 Out. 1
2
All
Fit.
MSEtr
MSEte
CTr 1
CTr 2
CTe 1
CTe 2
Top.
B
30444
3E-05
1E-02
0.9991
-
0.8382
-
4:13:01
W
75
1E-02
1E-02
0.4828
-
0.7178
-
4:17:01
A
2137
1E-03
1E-02
0.9681
-
0.8082
-
-
B
4446
2E-04
4E-03
0.9900
-
0.9307
-
4:13:01
W
155
6E-03
1E-03
0.6510
-
0.9742
-
4:14:01
A
2150
7E-04
1E-03
0.9671
-
0.9716
-
-
B
498
2E-03
7E-03
0.9955
0.8090
0.8501
0.9452
4:10:02
W
66
2E-02
1E-02
0.1814
0.2875
0.5621
0.8778
4:04:02
A
257
4E-03
8E-03
0.9131
0.6938
0.8114
0.8620
-
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
407
Table 13.3(b) Results obtained with SADE-NN-2 Out. 1
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
2
All
B W A B W A B W A
Fit. 37949 183 1988 6070 79 2270 538 81 264
MSEtr 3E-05 5E-03 1E-03 2E-04 1E-02 7E-04 2E-03 1E-02 4E-03
MSEte 1E-02 1E-02 1E-02 1E-03 5E-03 1E-03 6E-03 1E-02 8E-03
CTr 1 0.9993 0.8451 0.9724 0.9933 0.2905 0.9680 0.9742 0.7124 0.9200
CTr 2 0.8728 0.3418 0.6903
CTe 1 0.8183 0.7366 0.8132 0.9720 0.8912 0.9725 0.8698 0.4906 0.8220
CTe 2 0.9325 0.8643 0.8546
Top. 4:18:01 4:16:01 4:05:01 4:07:01 4:10:02 4:09:02 -
CTe 2 0.9518 0.8852 0.8764
Top. 4:07:01 4:08:01 4:03:01 4:07:01 4:17:02 4:06:02 -
Table 13.3(c) Results obtained with hSADE-NN Out. 1
2
All
B W A B W A B W A
Fit. 38280 229 2357 4140 71 2275 509 63 270
MSEtr 3E-05 4E-03 9E-04 2E-04 1E-02 7E-04 2E-03 2E-02 4E-03
MSEte 1E-02 7E-03 1E-02 6E-04 5E-03 2E-03 8E-03 1E-02 7E-03
CTr 1 0.9993 0.9111 0.9777 0.9901 0.2840 0.9687 0.9951 0.2565 0.9079
CTr 2 0.8144 0.2914 0.7135
CTe 1 0.8064 0.8272 0.8183 0.9883 0.8939 0.9708 0.8279 0.3217 0.8272
Fig. 13.12 shows that for both DE based variants, the model where a single output is considered is more close to the experimental data. Further, it can be observed that hSADE-NN predictions are closer to the expected value, which points out that it is better at finding optimal models when only F1 is taken into consideration. A point by point comparison for the two outputs is presented in Figs. 13.13 and 13.14, in order to assess the differences between the three DE based variants. All the three variants tend to generate good predictions for x output. For the F1 output, the differences between predictions and experimental data are bigger. This indicates that F1 is more difficult to model, and its dynamics cannot be entirely captured by the determined
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
408
models. On the other hand, the x changes are more accurately reproduced by the ANN models. Experimental
(a)
All outputs
one output
0.9 0.8 0.75 0.7 0.65 0.6 1
2
3
Experimental
(b)
4 5 EXEMPLARS
All outputs
6
7
8
7
8
One output
0.9 0.85 0.8 F1
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
F1
0.85
0.75 0.7 0.65 0.6 1
2
3
4
5
6
Exemplars Fig. 13.12 Comparison between the predictions generated by the models with a single output (F1) and multiple outputs for (a) SADE-NN and (b) hSADE-NN
Similar to the previous case, the results obtained for modeling the siloxane-siloxane copolymer synthesis by the applied methodologies (SADE-NN-1, SADE-NN-2 and hSADE-NN) are better than those reported in Curteanu and Cazacu (2007), in terms of the accuracy of the predictions of copolymer composition and reaction conversion.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Applications of DE in Polymerization Reaction Engineering
409
60 50
X
40 30
10 0 1
2
3
4
5
6
7
8
EXEMPLARS Experimental
SADE-NN-1
SADE-NN-2
hSADE-NN
Fig. 13.13 Comparison between experimental data and predictions generated by the three DE based variants for the x output 0.9 0.85 0.8 F1
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
20
0.75 0.7 0.65 0.6 1
2
3
4
5
6
7
8
EXEMPLARS Experimental
SADE-NN-1
SADE_NN-2
hSADE-NN
Fig. 13.14 Comparison between experimental data and predictions generated by the three DE base variants for the F1 output
13.6 Conclusions In this chapter, neuro-evolution was discussed in the context of chemical engineering (particularly, polymerization reactions) and DE algorithm. First, the main difficulties related to the modeling and optimization of
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
410
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
polymerization problems were laid down, followed by the principal aspects of neuro-evolution and how it can be used to solve specific aspects. In addition, three DE based variants in combination with ANNs were applied to model two polymerization engineering problems, namely, polyacrylamide based hydrogels synthesis and siloxane-siloxane copolymers synthesis. For these processes, DE variants were considered for modeling with one output at a time and multiple outputs together. Due to the dynamics of the processes, influence of each parameter and more difficulty of modeling some outputs than the others, modeling with single output provides better results. Although the more complex DE version (hSADE-NN) generally tended to provide the best average results, the best models were obtained with various variants, depending on the process characteristics. References Bhaskar V., Gupta S. K. and Ray A. K. (2000). Applications of multi-objective optimization in chemical engineering. Reviews in Chemical Engineering, 16, pp. 1-54. Bhutani N., Rangaiah G. P. and Ray A. K. (2006). First principles, data based and hybrid modeling and optimization of an industrial hydrocracking unit. Industrial and Engineering Chemistry Research, 45, pp. 7807-7816. Boozarjomehry R. B. and Svrcek W. Y. (2001). Automatic design of neural network structures. Computers & Chemical Engineering, 25, pp. 1075-1088. Chandra A. and Yao X. (2006). Ensemble learning using multi-objective evolutionary algorithms. Journal of Mathematical Modelling and Algorithms, 5, pp. 417-445. Curteanu S. (2004). Direct and inverse neural network modeling in free radical polymerization. Central European Journal of Chemistry, 2, pp. 113-140. Curteanu S. and Cazacu M. (2007). Neural networks and genetic algorithms used for modeling and optimization of the siloxane‐siloxane copolymers synthesis. Journal of Macromolecular Science, Part A: Pure and Applied Chemistry, 45, pp. 23-36. Curteanu S., Dragoi E. N., Leon F. and Butnariu C. (2014a). Artificial intelligence modelling methodologies applied to a polymerization process. 2014 International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH), Vienna, pp. 43-49. Curteanu S., Dumitrescu A., Mihăilescu C. and Simionescu B. (2008). Neural network modeling applied to polyacrylamide based hydrogels synthetized by single-step process. Polymer-Plastics Technology and Engineering, 47, pp. 1061-1071. Curteanu S., Dumitrescu A., Mihailescu C. and Simionescu B. C. (2009). The synthesis of polyacrylamide-based multi-component hydrogels: A neural network modeling. Journal of Macromolecular Science, Part A: Pure and Applied Chemistry, 46, pp. 368-380. Curteanu S., Leon F., Furtuna R., Dragoi E. N. and Curteanu N. (2010). Comparison between different methods for developing neural network topology applied to a
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Applications of DE in Polymerization Reaction Engineering
411
complex polymerization process. The 2010 International Joint Conference on Neural Networks (IJCNN), Barcelona, Spain. 1-8. Curteanu S., Suditu G., Buburuzan A. M. and Dragoi E. N. (2014b). Neural networks and differential evolution algorithm applied for modelling the depollution process of some gaseous streams. Environmental Science and Pollution Research, 21, pp. 12856-12867. Das S., Abraham A., Chakraborty U. K. and Konar A. (2009). Differential evolution using a neighborhood-based mutation operator. IEEE Transactions on Evolutionary Computation, 13, pp. 526-553. Das S. and Suganthan P. N. (2011). Differential evolution: A survey of the state-of-theart. IEEE Transactions on Evolutionary Computation, 15, pp. 4-31. Dash C. S. K., Dash A. P., Dehuri S., Cho S. B. and Wang G. N. (2013). DE+RBFNs based classification: A special attention to removal of inconsistency and irrelevant features. Engineering Applications of Artificial Intelligence, 26, pp. 2315-2326. Donate J. P., Li P., Sánchez G. G. and de Miguel A. S. (2013). Time series forecasting by evolving artificial neural networks with genetic algorithms, differential evolution and estimation of distribution algorithm. Neural Computing and Applications, 22, pp. 11-20. Dragoi E. N. and Curteanu S. (2016). The use of differential evolution for solving chemical engineering problems, Review in Chemical Engineering, 32, pp. 149-180. Dragoi E. N., Curteanu S. and Fissore D. (2012). Freeze-drying modeling and monitoring using a new neuro-evolutive technique. Chemical Engineering Science, 72, pp. 195-204. Dragoi E. N., Curteanu S. and Fissore D. (2013a). On the use of artificial neural networks to monitor a pharmaceutical freeze-drying process. Drying Technology, 31, pp. 7281. Dragoi E. N., Curteanu S., Galaction A. I. and Cascaval D. (2013b). Optimization methodology based on neural networks and self-adaptive differential evolution algorithm applied to an aerobic fermentation process. Applied Soft Computing, 13, pp. 222-238. Dragoi E.-N., Curteanu S., Cascaval D. and Galaction A.-I. (2016). Artificial neural network modelling of mixing efficiency in a split-cylinder gas-lift bioreactor for yarrowia lipolytica suspensions. Chemical Engineering Communications, 203, pp. 1600-1608. Du W., Leung S. Y. S. and Kwong C. K. (2015). A multiobjective optimization-based neural network model for short-term replenishment forecasting in fashion industry. Neurocomputing, 151, pp. 342-353. Durr P., Mattiussi C. and Floreano D. (2006). Neuroevolution with analog genetic encoding. In: Runarsson T., Beyer H. G., Burke E., Merelo-Guerv+Ýs J., Whitley L. & Yao X. (eds.) Parallel Problem Solving from Nature - PPSN IX. Springer Berlin / Heidelberg. Feoktistov V. (2006). Differential Evolution: In Search of Solutions, Berlin, Springer. Fister I., Suganthan P. N., Fister I., Jr., Kamal S. M., Al-Marzouki F. M., Perc M. and Strnad D. (2016). Artificial neural network regression as a local search heuristic for ensemble strategies in differential evolution. Nonlinear Dynamics, 84, pp. 895-914. Freeman J. A. and Skapura D. M. (1991). Algorithms, Applications, and Programming Techniques, Addison-Wesley Publishing, USA.
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
412
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
Gong W. Y., Cai Z. H. and Wang Y. (2014). Repairing the crossover rate in adaptive differential evolution. Applied Soft Computing, 15, pp. 149-168. Guo S.-M. and Yang C.-C. (2015). Enhancing differential evolution utilizing eigenvector-based crossover operator. IEEE Transactions on Evolutionary Computation, 19, pp. 31-49. Islam M. and Yao X.(2008). Evolving artificial neural network ensembles. In: Fulcher J. & Jain L. (eds.) Computational Intelligence: A Compendium. Springer Berlin / Heidelberg. Jung J. Y. and Reggia J. (2008). The automated design of artificial neural networks using evolutionary computation. In: Yang A., Shan Y. & Bui L. (eds.) Success in Evolutionary Computation. Springer Berlin / Heidelberg. Kazakov A. L. and Lempert A. A. (2015). On mathematical models for optimization problem of logistics infrastructure. International Journal of Artificial Intelligence™, 13, pp. 200-210. Kiranyaz S., Ince T., Yildirim A. and Gabbouj M. (2009). Evolutionary artificial neural networks by multi-dimensional particle swarm optimization. Neural Networks, 22, pp. 1448-1462. Kişi Ö. (2013). Evolutionary neural networks for monthly pan evaporation modeling. Journal of Hydrology, 498, pp. 36-45. Kohl N. and Miikkulainen R. (2009). Evolving neural networks for strategic decisionmaking problems. Neural Networks, 22, pp. 326-337. Lee M. H., Han C. and Chang K. S. (1999). Dynamic optimization of a continuous polymer reactor using a modified differential evolution algorithm. Industrial and Engineering Chemistry Research, 38, pp. 4825-4831. Leeghim H., Seo I. H. and Bang H. (2008). Adaptive nonlinear control using input normalized neural networks. Journal of Mechanical Science and Technology, 22, pp. 1073-1083. Leon F., Piuleac C.G. and Curteanu S. (2010). Stacked neural network modeling applied to the synthesis of polyacrylamide-based multicomponent hydrogels. Macromolecular Reaction Engineering, 4, pp. 537-636. Liao G. C. (2014). Hybrid improved differential evolution and wavelet neural network with load forecasting problem of air conditioning. International Journal of Electrical Power and Energy Systems, 61, pp. 673-682. Liu H., Cai Z. and Wang Y. (2010). Hybridizing particle swarm optimization with differential evolution for constrained numerical and engineering optimization. Applied Soft Computing, 10, pp. 629-640. Liu W., Song H., Liang J. J., Qu B. and Qin A. K. (2014). Neural network based on selfadaptive differential evolution for ultra-short-term power load forecasting. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Liu Y. (2006). How to stop the evolutionary process in evolving neural network ensembles. In: Jiao L., Wang L., Gao X-B., Liu J. & Wu F. (eds.) Advances in Natural Computation. Springer Berlin / Heidelberg. Macsim I. F., Dragoi E. N., Cazacu M. and Curteanu S. (2015). Neural network modeling of the preparation process of a siloxane-organic polyazomethine. Journal of Applied Polymer Science, 132.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Applications of DE in Polymerization Reaction Engineering
413
Mareci D., Dragoi E. N., Bolat G., Chelariu R., Gordin D. and Curteanu S. (2015). Modelling the influence of pH, fluoride, and caffeine on the corrosion resistance of TiMo alloys by artificial neural networks developed with differential evolution algorithm. Materials and Corrosion-Werkstoffe Und Korrosion, 66, pp. 982-994. Moghri M. and Dragoi E.-N. (2015). Effect of various material parameters on barrier properties of high-density polyethylene/polyamide 6/clay nanocomposites Experimental and prediction using artificial neural networks. Journal of Elastomers and Plastics, doi: 10.1177/0095244315618700 Neri F. and Tirronen V. (2010). Recent advances in differential evolution: a survey and experimental analysis. Artificial Intelligence Review, 33, pp. 61-106. Noor R. A. M., Ahmad Z., Don M. M. and Uzir M. H. (2010). Modelling and control of different types of polymerization processes using neural networks technique: A review. Canadian Journal of Chemical Engineering, 88, pp. 1065-1084. Oyebode O. and Adeyemo J. (2014). Reservoir inflow forecasting using differential evolution trained neural networks. In: Tantar AA et al. (Editors), EVOLVE - A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation V. Springer, Berlin, pp. 307-319. Pant M., Thangaraj R., Abraham A. and Grosan C. (2009). Differential evolution with Laplace mutation operator. IEEE Congress on Evolutionary Computation (CEC'09), Trondheim. IEEE, 2841-2849. Piotrowski A. P. (2013). Adaptive memetic differential evolution with global and local neighborhood-based mutation operators. Information Sciences, 241, pp. 164-194. Piotrowski A. P., Napiorkowski M. J., Kalinowska M., Napiorkowski J. J. and Osuch M. (2016). Are evolutionary algorithms effective in calibrating different artificial neural network types for streamwater temperature prediction? Water Resources Management, 30, pp. 1217-1237. Pirdashti M., Curteanu S., Kamangar M. H., Hassim M. H., Amid M. A. (2013): Artificial neural networks: applications in chemical engineering. Reviews in Chemical Engineering, 29, pp. 205-239. Pirdashti M., Movagharnejad K., Curteanu S., Dragoi E. N. and Rahimpour F. (2015). Prediction of partition coefficients of guanidine hydrochloride in PEG-phosphate systems using neural networks developed with differential evolution algorithm. Journal of Industrial and Engineering Chemistry, 27, pp. 268-275. Precup R.-E., David R.-C., Petriu E. M., Preitl S. and Radac M.-B. (2013). Fuzzy logicbased adaptive gravitational search algorithm for optimal tuning of fuzzy-controlled servo systems. IET Control Theory & Applications, 7, pp. 99-107. Priddy K. and Keller P. (2005). Artificial Neural Networks: An Introduction, Washington, SPIE Press. Raha S., Majumdar S. and Mitra K. (2004). Effect of caustic addition in epoxy polymerization process: A single and multi-objective evolutionary approach. Macromolecular Theory and Simulations, 13, pp. 152-161. Ramírez-Ortegón M. A., Märgner V., Cuevas E. and Rojas R. (2013). An optimization for binarization methods by removing binary artifacts. Pattern Recognition Letters, 34, pp. 1299-1306. Rangaiah G.P. (Editor) (2009), Multi-objective optimization: Techniques and applications in chemical engineering, Vol. 1 in the Advances in Process Systems Engineering, World Scientific, Singapore.
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
414
9in x 6in
b2817-ch13
E.-N. Dragoi and S. Curteanu
Rangaiah, G. P. (Editor) (2010). Stochastic global optimization techniques and applications in chemical engineering: Techniques and applications in chemical engineering, Vol. 2 in the Advances in Process Systems Engineering, World Scientific, Singapore. Rangaiah G. P. and Bonilla-Petriciolet A. (Editors) (2013). Multi-objective optimization in chemical engineering: Developments and applications, John Wiley. Rangaiah G. P., Sharma S. and Sreepathi B. K. (2015). Multi-objective optimization for the design and operation of energy efficient chemical processes and power generation, Current Opinion in Chemical Engineering, 10, pp. 49-62. Rasdi Rere L. M., Fanany M. I. and Arymurthy A. M. (2016). Metaheuristic algorithms for convolution neural network. Computational Intelligence and Neuroscience, Online at: https://www.hindawi.com/journals/cin/2016/1537325/abs/ Sarker R. A., Elsayed S. M. and Ray T. (2014). Differential evolution with dynamic parameters selection for optimization problems. IEEE Transactions on Evolutionary Computation, 18, pp. 689-707. Sharma S. and Rangaiah G. P. (2013). An improved multi-objective differential evolution with a termination criterion for optimizing chemical processes. Computers & Chemical Engineering, 56, pp. 155-173. Srinivas M. and Rangaiah G. P. (2007a). Differential evolution with tabu list for global optimization and its application to Phase Equilibrium and Parameter Estimation Problems. Industrial and Engineering Chemistry Research, 46, pp. 3410-3421. Srinivas M. and Rangaiah G. P. (2007b) Differential evolution with tabu list for solving non-linear and mixed-integer non-linear programming problems, Industrial and Engineering Chemistry Research, 46, pp. 7126-7135. Storn R. (2008). Differential evolution research - trends and open questions. In: Chakraborty U. (ed.) Advances in Differential Evolution. Berlin / Heidelberg: Springer. Thangaraj R., Pant M. and Abraham A. (2009). A simple adaptive differential evolution algorithm. World Congress on Nature & Biologically Inspired Computing (NABIC 2009), Coimbatore. IEEE, 457-462. Thangraj R., Pant M., Abraham A., Deep K. and Snasel V. (2010). Differential evolution using a localized Cauchy mutation operator. 2010 IEEE International Conference on Systems Man and Cybernetics (SMC). IEEE, 3710-3716. Tian Y., Zhang J. and Morris J. (2001). Modeling and optimal control of a batch polymerization reactor using a hybrid stacked recurrent neural network model. Industrial and Engineering Chemistry Research, 40, pp. 4525-4535. Volna E. (2010). Neuroevolutionary optimization. International Journal of Computer Science Issues, 7, pp. 31-37. Wang H., Rahnamayan S., Sun H. and Omran M. G. (2013). Gaussian bare-bones differential evolution. IEEE Transactions on Cybernetics, 43, pp. 634-647. Wang J. and Guo Q. (2013). Kernel principal component analysis: Radial basis function neural networks-based soft-sensor modeling of polymerizing process optimized by cultural differential evolution algorithm. Instrumentation Science and Technology, 41, pp. 18-36. Wang L., Zeng Y. and Chen T. (2014a). Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Systems with Applications, 42, pp. 855-863.
Differential Evolution in Chemical Engineering
9in x 6in
b2817-ch13
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 06/03/17. For personal use only.
Applications of DE in Polymerization Reaction Engineering
415
Wang. and Zhao S. G. (2013). Differential evolution algorithm with self-adaptive population resizing mechanism. Mathematical Problems in Engineering. Online at: https://www.hindawi.com/journals/mpe/2013 /419372/abs/. Wang Y., Li H., Huang T. W. and Li L. (2014b). Differential evolution based on covariance matrix learning and bimodal distribution parameter setting. Applied Soft Computing, 18, pp. 232-247. Xin Y. (1999). Evolving artificial neural networks. Proceedings of the IEEE, 87, pp. 1423-1447. Yardimci A. (2009). Soft computing in medicine. Applied Soft Computing, 9, pp. 10291043. Yoon Y. H. and Nelson J. H. (1984). Application of gas adsorption kinetics I. A theoretical model for respirator cartridge service life. American Industrial Hygiene Association Journal, 45, pp. 509-516. Yu W. J., Shen M., Chen W. N., Zhan Z. H., Gong Y. J., Lin Y., Liu O. and Zhang J. (2014). Differential evolution with two-level parameter adaptation. IEEE Transactions on Cybernetics, 44, pp. 1080-1099. Zhang L., Li H. and Feng D. (2014). ODE-LM: A hybrid training algorithm for feedforward neural networks. Advances in Intelligent Systems and Computing. Zhu W., Tang Y., Fang J. A. and Zhang W. B. (2013). Adaptive population tuning scheme for differential evolution. Information Sciences, 223, pp. 164-191. Zielinski K. and Laur R. (2008). Stopping criteria for differential evolution in constrained single-objective optimization. In: Chakraborty U. (ed.) Advances in Differential Evolution. Berlin / Heidelberg: Springer.
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-index
Index A
Aspen HYSYS, 131, 141, 294 Aspen Plus, 232, 276 augmented Lagrangian, 47, 50 azeotrope of ethanol-water, 291 azeotropic composition, 291 azeotropic mixture, 248
a posteriori decision-making methods, 354 absolute deviation, 83 absorber, 143 acetone-butanol-ethanol fermentation, 271 acetone/butanol/ethanol/water, 261 acid gases, 143 activation functions, 388 activity data, 304 adaptation, 14 adiabatic efficiency, 292 adiabatic thermal efficiency, 300 aerobic fermentation, 393 algorithm reliability, 209 alkylation process, 28 alkylation process operation, 60 amine absorption process, 24, 130, 141 ANN (Artificial Neural Network) node, 382 annual operating cost, 143, 171 annualization of capital cost, 172 Ant Colony Optimization (ACO), 9 approach temperature, 19 architecture, 384 artificial bee colony, 321 flow chart, 337 artificial neural network, 377, 382 structure, 383 training, 382 training performance, 384 Aspen custom modeler (ACM), 301 Aspen economic analyzer, 246, 254
B back propagation, 383 bare module costs, 303 batch chemical plant, 59 batch fermentation, 271 batch operation, 328 beer fermentation, 60 bees algorithm, 10 benchmark functions modified N-dimensional test function, 121 benchmark problem, 23, 91, 93, 101, 118, 220 difficult functions, 101 new benchmark problem, 119 moderate functions, 101 benchmarks, 378 bi-functional catalyst, 61 bi-functional catalyst blending, 59 bi-level optimization, 352–354, 366, 371 bi-level linear optimization, 366 biases, 388 binary decision variables, 265 binary feed, 294 binary strings, 9 binomial crossover, 132, 163
417
page 417
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
418 bio-reactors, 56 biobutanol purification design parameters, 284 performance indexes, 284 biodegradation of phenol, 57 bioethanol, 291 bioethanol purity, 306 bioethanol recovery distillation, 291 extraction, 291 gas/steam stripping, 291 membrane separation, 291 polyacrylamide membrane, 293 polyvinyl alcohol membrane, 293 reverse osmosis (RO), 293 third generation feedstock, 293 bioethanol separation, 25 biofuels, 290 biological process optimization, 356 biomass concentration profile, 346 bioprocess, 346 bisection method, 202 bit strings, 37 black boxes, 381 black-box problems, 7, 36 boundary violation, 139 boundary violation check, 98 branch-and-bound methods, 194 butanol production, 270 see also biobutanol purification butyl acetate production, 202 by-products formation, 71 C 5-component mixture, 250 Candida mogii yeast, 321, 325 capital cost, 17, 246 carbon dioxide (CO2 ), 141, 290 carbon dioxide capture, 141 catalyst activity, 77–78 catalyst replacement, 73 catalytic gas oil reaction, 57 catalytic hydrogenation, 322 Cauchy distributions, 134 Chebychev polynomial fitting, 36
9in x 6in
b2817-index
Index chemical and phase equilibrium, 187 chemical engineering applications, 21 chemical engineering plant cost index (CEPCI), 143, 303 chemical equilibration procedure, 194 chemical equilibrium constraints, 192 chemical potentials, 191 chemical reaction engineering, 23, 70 chi-squared distribution function, 137 chi-squared test, 137 child population, 164 chromosome, 9 classic DE, 133 classification of optimization methods, 4 clonal selection, 398 clustering method, 110 column convergence, 306 column efficiency, 302 column section, 234 column section functionality, 241 column section recombination, 235 combinatorial problems, 37 combined population, 164 common objective functions, 16 comparable minima, 119 complex columns, 234 composition variables, 190 compromise programming, 11 computational efficiency, 100, 181 computational effort, 181, 250 computational fluid dynamics (CFD), 73 computational time, 254–255, 308 computer programs, 27 concentration of reactants, 82 condition number, 273, 281 constrained dominance, 165, 267 constrained-dominance criteria, 140 constrained optimization, 5, 352 constraint, 3, 77, 144, 26, 306 constraint violation, 140 continuous parameters, 233 continuous problems, 37 continuous stirred tank reactor (CSTR), 57, 59 continuous variables, 171, 176
page 418
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Index control parameters, 40, 243 control properties, 273, 278 control test, 276 control vector parameterization (CVP) strategy, 329 controllability, 272 controller design for the ALSTOM gasifier, 58 convergence check, 40 convergence criteria, 40 convergence metric (CM), 135, 170 convergence time, 249 convergence to the optimal solution, 381 conversions, 78 convolution ANN, 387 cooling water, 19 copolymer composition, 406 copolymerization reactor, 79, 397 corrosion resistance, 395 cost estimation methods, 316 cost of manufacture (COM), 294 cost of raw materials, 303 cost of replacing membrane sheets, 303 costing of equipment, 301 counter-current flow, 166, 300 CPU time, 100, 181 cross-flow, 301 cross-linked hydrophilic polymers, 293 crosslinking reaction, 402 crossover, 40, 49, 71, 243, 358 crossover operation, 9, 332 crossover probability, 12, 40–41, 243, 275 crossover probability constant, 333 crowding distance(s), 140, 165 crowding distance calculations, 149, 164, 267 crude distillation unit, 173 cuckoo search, 10 cycle length prediction, 82 D data collection, 380 data pre-processing, 401 DE with tabu list (DETL), 55, 93 DE/best/1, 163
9in x 6in
b2817-index
419 DE/best/2, 163 DE/rand-to-best/1, 163 DE/rand/1, 132, 163 DE/rand/2, 163 decision variable(s), 3, 37, 70, 76–77, 144, 169, 277, 305 optimization variable, 329 decision-making maximization problem, 366 decoding, 390 degasser column, 295 degrees of freedom, 264, 299 DeMAT software, 41 depollution process of gaseous streams, 395 derivative-based methods, 5 derivative-free methods, 5 design formulation, 364 design method, 231 design mistake, 249 design objective, 231 design parameters, 241 design variables, 249 deterministic approaches, 75, 242 deterministic optimization methods, 7 di-ethanolamine (DEA), 143 diesel catalytic cracking, 58 difference vector, 40 differential evolution, 36, 70, 92, 187, 323, 378 amplification factor, 95 arithmetic recombination, 50 base vector selection, 43 classic DE, 11 classic DE algorithm, 37 cultured DE, 47 discrete DE, 48 dither, 45 estimation of distribution algorithm, 50 evaluation step, 94 gene swap, 50 generation step, 94 hybrid DE, 82 hybridization, 379
page 419
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
420 improve, 379 modifications, 379 immunology algorithms, 47 initialization, 43 jitter, 45 modified DE, 76 modified differential evolution, 93 mutation algorithm, 48 mutation factor, 41, 45 mutation operation, 43 neighborhood search, 53, 54 new generation, 52 NHDE, 358 optimization problems, 378 parameters, 36, 41 population size, 41 selection, 49 selection step, 94 software, 42 swarm DE, 92 trigonometric DE, 48 differential evolution with tabu list, 91, 94, 131, 261, 265 DETL-E, 95 DETL-G, 95 flow chart of DETL-E, 96 flow chart of DETL-G, 97 difficulties in modeling, 379 diffusive transport equation, 327 diphenyl carbonate (DPC), 269 diphenl carbonate production, 268 design parameters, 279 free-phosgene process, 269 performance indexes, 279 phosgene process, 269 trans-esterification reaction, 269 direct encoding, 388 direct search methods, 99, 219 discrete parameters, 233 discrete variables, 176, 276 distillation module, 299 distillation-vapor permeation process, 295 diversification and intensification strategies, 339 dividing-wall columns, 240, 264, 271
9in x 6in
b2817-index
Index duality theory, 352, 363 dynamic behavior, 264 dynamic optimization, 26, 321, 397 dynamic optimization problems, 323 dynamic penalty, 323 dynamic properties, 280 E ε-constraint method, 11, 161 Easom, 102 eco-efficient option, 261 eco-indicator-99, 272, 280 economic benefits, 263 efficiency of stages, 143 electric motor driven, 300 elitist non-dominated sorting genetic algorithm, 161 EMILiO, 363 emission factor, 304 emission of greenhouse gases, 244 EMOO program, 146 empirical models, 381 encoding, 385 direct, 385 implicit, 385 indirect, 385 energy balance, 18 energy efficiency, 160 energy requirement, 313 equivalent energy requirement, 312 specific energy requirement, 312 energy savings, 77 engineering techniques, 270 ensembles, 385 environmental benefits, 263 environmental objective, 272 equality constraints, 37, 141, 336, 352 equality-constrained optimization, 367 equilibrium equations, 75 equipment sizing, 301 escaping mechanism, 94 escaping strategies, 224 esterification reaction, 219 ethanol dehydration, 291 ethanol production rate, 296
page 420
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-index
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Index ethanol recovery, 291 ethanol recovery ratio, 296 ethanol-water-CO2 system, 315 Euclidean distance, 98, 136, 170 Eulerian methods, 73 evaporator, 57 evolution strategy, 356 evolutionary algorithms, 75, 353, 358, 360–361, 364 evolutionary methods, 8–9 evolutionary optimization, 354 excess Gibbs free energy approach, 112 experimental knowledge, 382 exploitation strategies, 8 exponential crossover, 163 extraction agent, 271 extractive distillation, 269 extractive-dividing wall column (E-DWC), 310 vapor recompression assisted E-DWC, 312 F feasibility approach, 140, 165, 267 feasible search space, 265 fed-batch fermentation, 59, 341 two-feed operation, 342 fed-batch bioreactor, 62 fed-batch fermentor, 61 fed-batch operating condition, 325 fed-batch operation, 344 fed-batch reactor, 25, 321, 325 feed and concentration profiles, 338 feed location, 251 feed stage, 306 fermentation, 291 fermentation broth, 292, 294 fermentation model, 330 fermentation process, 16, 322 fermentation system, 340 fermentation process for ethanol, 60 fermentor with cell recycling and extractor, 59 film resistances, 302 filter coefficient, 138
421 firefly algorithm, 10 first generation feedstock, 291 first in first out, 139 first order response, 274 fitness function, 389 fitness value, 335 fixed bed reactors, 73 fixed capital investment, 303 floating-point numbers, 37 floating-point values, 266 flow chart of differential evolution, 38, 332 flow rate, 79 fluid power, 300 fluid velocity, 72 fluidization, 72 fluidized bed reactors, 72 flux balance analysis (FBA), 372 flux of xylitol, 327 flux variability analysis, 373 food versus fuel debate, 291 FORTRAN code for DE, 99 FORTRAN code for TS, 99 FORTRAN codes, 197 fouling factors, 302 free radical polymerization of styrene, 398 freeze-drying, 394 fuel blends, 291 fuel-grade ethanol (FGE), 291 fugacity approach, 112 fugacity coefficient, 112 function evaluation step, 133 fuzzy programming, 366 G GAMS environment, 362, 370 gas-phase reactions, 78 gene-knockout problems, 371 generalized reduced gradient, 5 generating methods, 353 generational distance (GD), 135 modified GD, 136 genetic algorithms (GA), 9, 355, 360 genetic engineering techniques, 270 genome-scale metabolism, 363 genotype, 389
page 421
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
422 Gibbs energy minimization, 186 Gibbs free energy, 109, 112, 189 Gibbs free energy minimization, 191 global maximum, 6 global methods, 6 global optimization, 5 global optimization methods, 7 global optimum, 6, 75, 347 glycerol bioconversion, 324 goal programming, 11 Goldstein and Price, 102 gradient search optimization, 5 gradient-based methods, 219 grassroots cost, 303 greedy criterion, 40 green product alternative, 261 green route, 269 greenhouse gas (GHG), 290 Griewank, 103 growth-coupled production strain, 363 decision-making problem, 364
9in x 6in
b2817-index
Index Himmelblau function, 40 modified Himmelblau, 103 holdup, 277 homotopy continuation algorithms, 194 hSADE-NN, 388 hybrid algorithms, 197 hybrid design method, 249 hybrid differential evolution, 93 ant direction hybrid differential evolution, 93 hybrid distillation-vapor permeation, 295 hybrid intensified process, 275 hybrid methods, 51 hybrid separation processes, 292 hybrid stochastic algorithm, 261 hybrid stripper-membrane process, 292 hydrogels synthesis, 402 hydrogen sulfide, 141 hydrophilic membranes, 293 hydrophilic pervaporation (PV) modules, 292 hydrophobic PV modules, 292
H harmony search, 10 Hartmann 3 variables, 102 heat and mass transfer correlations, 75 heat capacity, 166 variable heat capacity, 168, 172 heat exchanger network (HEN) retrofitting, 17, 24, 62, 159 heat exchanger network design, 61 heat integration, 271, 298 heat transfer area, 302 heat transfer coefficient, 143, 166, 173, 394 heat transfer enhancements, 160 heat transfer rate equation, 18 heating medium, 19 HEN retrofit case studies, 171 HEN retrofitting methodology, 166 HEN structure, 166 Hessian matrix, 197 hexyl-acetate, 271 hidden layers, 388 high flow rates, 72, 74
I ideal stages, 299 ideal thermodynamic model, 276 IMODE flowchart, 142 IMODE algorithm, 139 IMODE program, 141, 306 improvement-based termination criteria, 151 IMSL library, 197 IMSL software, 99–100 incremental improvements, 388 individual generation step, 133 industrial reactors, 74 inequality constraints, 37 infeasible regions, 333 infeasible solutions, 249 inherent safety, 263 inhibition effect, 344 initial population, 37 initialization, 71 initiation, 243 inner optimization problem, 355, 361–363
page 422
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Index integer variables, 171 intensified distillation systems, 238 intensified process, 264, 283 intensified process design, 264 intensified process optimization, 265 intensified separation processes, 25 intensified sequences, 246 intensified simple columns, 240 interfacing between MS Excel and Aspen HYSYS, 145 interval Newton technique, 110, 194 intracellular xylitol, 326 inverse generational distance, 135 investment cost, 18, 169 iterative dynamic programming, 79 K kernel principal component analysis, 399 kinetic analysis, 82 kinetic model, 325 kinetic modeling, 23, 74 kinetic parameters, 82, 328 Kuhn–Tucker algorithms, 353 Kuhn–Tucker theory, 352 L lactic acid concentration, 323 lactic acid production, 59 lactic acid recovery, 62 Lagrangian approach, 73 latent heat of condensation, 19 Le Chatelier’s principle, 80 leapfrogging technique, 137 learning rate parameter, 383 learning rules, 384 Lehmer mean, 135 length over radius, 80 lignocellulosic materials, 291 linear optimization, 373 LINPACK benchmark program, 100, 202 liquid height, 277 liquid-liquid equilibrium, 203 liquid-liquid extraction, 267, 271 liquid-liquid extraction column, 271 local maxima, 6
9in x 6in
b2817-index
423 local methods, 7 local minimum, 6, 335 local optimization, 99 local search methods, 6 log mean temperature difference, 18, 302 logarithmic mean temperature difference (LMTD), 143 low pressure drops, 72 lumped method, 82 M manipulated variables, 274 Margules solution model, 226 mass and energy balances, 4 material of construction, 302 MATLAB program, 159 maximum number of generations (MNG), 12, 130, 162, 169, 244 maximum number of successive generations, 95 maximum singular value, 273 maximum spread, 135 McCabe-Thiele method, 231 mean squared error (MSE), 389 mechanical efficiency, 300 membership function, 366 membrane areas, 305 membrane costs, 312 membrane durability, 80 membrane performance, 303 membrane permeance, 301 membrane pervaporation, 292 membrane reactor, 78 membrane thickness, 79 MESH, 276 metabolic engineering, 353, 363 metabolic networks, 26, 352, 363 metaheuristics, 8 methane, 141 methyl tert-butyl ether (MTBE) reaction, 202 Metropolis criterion, 10 microbiological conversion of xylose, 322 Microsoft visual studio, 301 migration operation, 358, 360
page 423
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
424 min-max normalization, 393 minimum approach temperature (MAT), 166 minimum singular value, 273 mixed-integer non-linear programming, 242 mixed-matrix membranes (MMMs), 293 MODE algorithm parameters, 165 model building, 54 model performance, 401 model-based optimization, 363 modeling, 377 molecular steps, 74 monitoring, 395 Monod saturation constants, 326 MOO problem formulation, 307 moving bed reactors, 73 MS Excel, 23, 131, 139 multi-modal, 6 multi-layer perceptron (MLP), 386 multi-objective differential evolution (MODE), 129, 159, 244, 294 multi-objective differential evolution (MODE) algorithm, 162, 251 multi-objective optimization, 84, 160, 353 multi-objective optimization problem, 341 multi-objective performance metrics, 135, 148 multi-reaction conditions, 210 multi-stage compressor, 298 multi-start, 6 multicomponent distillation alternatives, 233 multifunctional reactors, 77 multiple feed streams, 322 multiple thermodynamic models, 205 multistage dynamic optimization, 79 multivariable optimization, 5 mutant amplification factor, 332 mutant individual, 12, 164 mutant vector, 40, 332 mutation, 39, 71, 243, 358 mutation factor, 12, 45, 243, 275 mutation operation, 9 mutation principle, 391
9in x 6in
b2817-index
Index mutation scheme in DE, 356 mutation strategies, 391 N natural gas, 141 natural gas sweetening, 129, 141 natural selection theory, 37 Navier-Stokes equations, 73 NDS program in MS Excel, 149 Nelder–Mead simplex method, 219 nested hybrid differential evolution (NHDE), 354, 358 neuro-evolutive techniques, 377 NHDE algorithm, 362 flowchart of the NHDE, 359 nodal representation, 166 nodes, 166 non-convex objective functions, 283 non-dominated solutions, 10, 165, 170, 176, 179, 294 quality of, 148 non-dominated sorting, 164, 267 non-dominated sorting genetic algorithm, 324 non-dominating sorting (NDS) program, 149 non-random two liquid (NRTL), 110 non-reactive mixtures, 188 non-sharp separation, 240 non-sharp splits, 234 nonlinear optimization, 356 nonlinear programming, 5 normal boundary intersection, 11 normal distribution, 135 normalization, 393 normalization of decision variables, 99 normalized decision variable, 133 normalized decision variable space, 164 normalized normal constraint, 11 NRTL model, 220 NRTL-HOC model, 279 NRTL-RK, 296 NSGA-II program in MATLAB (NGPM), 159, 169 number of decision variables, 173
page 424
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
b2817-index
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
Index number of exchangers, 169 number of function evaluations (NFE), 100 number of ideal stages, 308 number of neurons, 388 number of optimization variables, 250 number of sub-sections in reactor, 80, 82 number of thermal couplings, 238 number of trays, 247 number of tubes, 81 numerical derivatives, 219 numerical performance, 347 O O’Connell correlation, 302 objective function, 3–4, 76–77, 83, 144, 265 objective function space, 135 objectives, 304 operating conditions, 324, 380 operating cost, 18, 172 operating labour, 303 opposition based learning, 393 optimal control of fed-batch bioreactors, 321, 323 optimal control parameters, 392 optimal design, 231, 249 optimal feeding profile, 324 optimal operating conditions, 77, 377 optimal overflow, 231 optimal solutions, 21 optimization, 3, 378 optimization method, 336 optimization problem, 4 optimization variables, 255 optimizing polymerization processes, 379 optimum number of plates, 231 OptKnock, 363 OptORF, 363 OptReg, 363 OptStrain, 363 original thermally coupled distillation, 237 outer optimization problem, 355, 362–363
425 overall heat transfer coefficient, 302 oxidation of p-xylene, 57 P parallel-computing’ toolbox, 181 parameter estimation, 13, 54 parameter estimation problems, 37 parameter estimation problems in fermentation, 56 parameter tuning, 101, 201 parent population, 164 Pareto optimal solutions, 84, 244, 370 Pareto-optimal front, 10, 353, 381 best Pareto-optimal front, 169 known Pareto-optimal front, 148, 169 Pareto ranks, 140 particle swarm optimization (PSO), 321 flow chart, 334 penalty function, 329 penalty function approach, 346 performance criterion, 83 performance metrics, 131, 135, 170, 180, 199, 294 performance profile, 199 permeability coefficient, 327 perturbation vector, 132 phase equilibrium calculations, 186–188, 213 phase stability, 186, 225 phase stability analysis, 186, 193, 194 modified stability criterion, 194 necessary and sufficient condition, 193 phase stability problems, 23, 92, 93, 109, 114, 193 PI technologies, 263 Classification of PI equipment and methods, 262 inherent limitations, 264 plant operating hours, 304 plug flow, 74 PM program, 150 polyazomethine, 400 polycarbonate, 268
page 425
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
426 polycondensation reaction, 400 polymeric membranes, 293 polymerization, 377 continuous, 380 discontinuous, 380 initial conditions, 380 polymerization engineering, 396 polymerization reaction engineering, 26, 377 polysiloxanes, 405 polyvinylchloride polymerization, 399 population based algorithms, 378 advantages, 378 population diversity, 358 population initialization, 12 population size, 12, 41, 243 population-based methods, 8 power generation, 16 power plants, 141 pre-concentration of ethanol, 291 pre-heat train, 173 preference-based methods, 354 pressure, 78 pressure swing adsorption (PSA), 310 process benefits, 263 process control, 82 process design, 13 process development, 296 process duration, 330 process intensification, 238, 261 classification of PI equipment and methods, 262 process intensification (PI) inventions, 263 process modeling, 272 process optimization, 54, 82, 377 process synthesis, 261 process synthesis and design problems, 60 product quality, 380 production capacity, 296 production costs, 381 productivity, 71, 76 Propionibacterium shemanii, 393 pseudo code for DE, 39 purchase cost, 303
9in x 6in
b2817-index
Index Q quartile sequential plots, 214 quasi-Newton method, 197, 217, 219 quaternary mixture, 245 R R program, 164 RADFRAC model, 246 radial basis functions (RBF), 383, 386 random search, 390 randomization, 7 Rastrigin, 103 rate equations, 4 ReacKnock, 363 reaction engineering, 16 reaction kinetics, 77 reaction mechanism, 74 reaction rates, 78 reaction-separation steps, 270 reactive distillation, 270 reactive distillation columns, 263 reactive phase equilibrium, 187 reactive phase stability, 187 reactive phase stability and equilibrium problems, 187 reactive separations processes, 186 reactive systems, 185, 187–188, 190 reactive tangent plane distance function, 195 reactor design, 70, 82, 85 reactor efficiency, 85 reactor modeling, 72 reactor operating conditions, 23 reactor size, 80 reactors, 16 real-life applications, 378 reassignment of existing exchangers, 168 recurrent neural network, 400 regenerator, 143 reinforcement-learning problems, 384 relative error, 83 relative gain matrix, 274 reliability of stochastic methods, 199 residence time, 79 rigorous design, 233
page 426
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Index rigorous design methods, 232 RobustKnock, 365 Rosenbrock, 102 modified Rosenbrock function, 120 Runge-Kutta method, 331
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
S simulated annealing (SA), 7–8, 9–10, 71, 93, 193–194, 224–225 Saccharomyces cervisiae, 394 SADE-NN-1, 388 SADE-NN-2, 388 search space, 164, 233, 243 selection, 40, 71, 243, 358 selection operation, 9 selection pressure, 45 selectivity, 76 self-adaptation of DE parameters, 24, 134, 162 self-adaptive mechanism, 392 sensitivity analysis, 246, 394 separation cost of bioethanol from water, 291 separation process design, 244 separation processes, 16 sequential design method, 25, 241 sequential quadratic programming (SQP), 310 sharp separation, 240 sharp separation sequences, 235 sharp splits, 234 shell and tube heat exchanger design, 62 short-cut design, 246 short-cut methods, 232 Shubert, 102 side stream, 251 sieve tray, 302 significant heat transfer, 79 siloxane-siloxane copolymers, 377 siloxane-siloxane copolymers synthesis, 405 simple columns, 234 simultaneous optimization, 388 single column sections, 240 single point method, 8
9in x 6in
b2817-index
427 single-objective optimization, 4, 83 singular value decomposition, 273 soft sensor, 58 solvent design, 62 solvent makeup, 313 specific consumption rate, 327 specific growth rate, 325 maximum specific growth rates, 326 specific pressure range, 78 specific rate of formation, 327 specific uptake rate, 326 spherical reactor, 80 spread (SP), 135, 171 modified SP, 136 spread distribution (SPd), 170 spread extreme (SPe), 170 stability calculations, 208 stacks, 395 stationary points, 195 statistical analysis, 336 steady-state conditions, 74 steady-state detection, 130 steady-state detection technique, 137 steam turbine, 300 stiff systems, 331 stochastic algorithms, 210, 242, 331 stochastic global optimization methods, 323 stochastic methods, 212, 346 stochastic optimization, 355 stochastic optimization algorithms, 265 stochastic optimization methods, 7, 75 stochastic optimizer, 344 stochastic sampling, 110 stochastic universal sampling, 45 stopping criteria, 71, 201, 392 strain design problem, 370 strength Pareto evolutionary algorithm (SPEA), 161 structural matrix, 166, 171 styrene reactor, 60 sub-mixtures, 240 success rate, 100 successive quadratic programming, 5 sulfur dioxide oxidation, 58
page 427
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
428 supervised learning, 382 supplementary material, 256, 316, 362 support vector machines, 398 swarm-intelligence based algorithms, 333 swelling degree, 403 synthesis procedure, 233, 240, 241 systematic synthesis methodology, 234, 236 T tabu check, 23, 133, 267 tabu list, 23, 266 tabu list size, 95 tabu radius, 95 tabu search, 10, 92, 186, 187 tangent plane criterion, 109, 110 tangent plane distance function, 109, 194 target individual, 12, 163 target vector, 40 temperature, 77 temperature driving force, 298 termination criterion, 13, 24, 100, 130, 137, 162, 244 ternary mixtures, 235 tert-amyl methyl ether (TAME) synthesis, 202 theoretical guarantee, 75 thermal coupling, 235 thermally coupled configurations, 249 thermally coupled reactor, 77, 78 thermodynamic efficiency, 245, 250 thermodynamic equilibrium, 77 thermodynamic inefficiency, 235 thermodynamic property package, 296 thermodynamically equivalent structure, 235, 238 total annual cost, 272 total annualized capital cost, 245 total capital cost, 143 tournament ranking, 45 tournament selection, 49 training algorithm, 383 training data, 393 transformed composition variables, 191 transformed Gibbs free energy, 205
9in x 6in
b2817-index
Index transformed molar Gibbs free energy of mixing, 190 transformed mole fractions, 192 transformed mole numbers, 191 transformed variables, 192 trial individual, 12, 163, 267 trial vector, 40 trickle bed reactor, 56 trivial solutions, 189 tubular reactor, 80 tuning process, 275 tunneling method, 110 turbulency, 72 U unbiased estimate of variance, 138 unbounded problems, 37 unconstrained minimization of free energy, 192 unconstrained minimization problem, 213 unconstrained optimization, 5 unconstrained optimization problem, 94 uniform distribution, 162, 357 uniformly distributed random numbers, 139 universal quasi-chemical (UNIQUAC), 110 unstable reactive mixtures, 195 unsupervised learning, 382 utility cost, 168 utility cost data, 303 V vapor permeation (VP), 25, 292 vapor transfer between columns, 238 vapor-liquid equilibrium, 203, 223, 232, 296 vapor-permeation membrane module, 300 variable heat capacity, 172 variance ratio, 138 vertex enumeration, 353 vinyl chloride monomer, 399 visual basic for applications (VBA), 131, 139, 294
page 428
April 20, 2017
17:4
Differential Evolution in Chemical Engineering
9in x 6in
Index W Wavelet ANN, 386 weighted sum, 11, 83, 161 weights, 384
Differential Evolution in Chemical Engineering Downloaded from www.worldscientific.com by 80.82.77.83 on 06/22/17. For personal use only.
X xylitol, 322 xylitol fed-batch fermentation, 331 xylitol fermentation process, 341 xylitol metabolism, 341 xylitol production, 25, 321
b2817-index
429 xylitol productivity, 323 xylitol yield, 338 Y yeast fermentation, 394 yeast growth, 326 yield, 76 Z Zakharov, 103
page 429
