D62172GC10_sg

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

bl a r fe

s

an r t n

no a s a h MySQL Developer n) ideฺTechniques v ฺ om t Gu c ฺ 21 den s t tuGuide sp Student S @ s i n h a t r t SQL-4305 Release 1.1 e ฺ s n u o o to m ( e an icens r T l et

uy g N

D62172GC10 Edition 1.0 January 2011 D63882

Copyright © 2008, 2009, 2011, Oracle and/or its affiliates. All rights reserved.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Disclaimer

Thi

This document contains proprietary information, is provided under a license agreement containing restrictions on use and disclosure, and is protected by copyright and other intellectual property laws. You may copy and print this document solely for your own use in an Oracle training course. The document may not be modified or altered in any way. Except as expressly permitted in your license agreement or allowed by law, you may not use, share, download, upload, copy, print, display, perform, reproduce, publish, license, post, transmit, or distribute this document in whole or in part without the express authorization of Oracle. The information contained in this document is subject to change without notice. If you find any problems in the document, please report them in writing to: Oracle University, 500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not warranted to be error-free. Sun Microsystems, Inc. Disclaimer This training manual may include references to materials, offerings, or products that were previously offered by Sun Microsystems, Inc. Certain materials, offerings, services, or products may no longer be offered or provided.Oracle and its affiliates cannot be held responsible for any such references should they appear in the text provided.

e

bl a r fe

Restricted Rights Notice If this documentation is delivered to the U.S. Government or anyone using the documentation on behalf of the U.S. Government, the following notice is applicable:

s

an r t n

o

U.S. GOVERNMENT RIGHTS The U.S. Government’s rights to use, modify, reproduce, release, perform, display, or disclose these training materials are restricted by the terms of the applicable Oracle license agreement and/or the applicable U.S. Government contract.

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Trademark Notice

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark licensed through X/Open Company, Ltd.

uy g N

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

This page intentionally left blank.

e

s

bl a r fe

o

an r t n

uy g N

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

Conventions Used in This Training Guide

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

This training guide uses certain typographical conventions: • TEXT IN THIS STYLE is used for SQL statements; database, table, and column names; program listings and source code; and environment variables. Example: “To reload the grant tables, use the FLUSH PRIVILEGES statement.”

Thi

Text in this style indicates input that you type in examples. Text in this style indicates the names of executable programs and scripts, examples being mysql (the MySQL command line client program) and mysqld (the MySQL server executable). • Text in this style is used for variable input for which you should substitute a value of your own choosing. • Filenames and directory names are written like this: “The global my.cnf file is located in the /etc directory.” • Character sequences are written like this: “To specify a wildcard, use the '%' character.” • Text in this style is used for emphasis. When commands are shown that are meant to be executed from within a particular program, the prompt shown preceding the command indicates which command to use. For example, shell> indicates a command that you execute from your login shell, and mysql> indicates a statement that you execute from the mysql client program: shell> type a shell command here mysql> type a mysql statement here The “shell” is your command interpreter. On Linux, this is typically a program such as sh, csh, or bash. On Windows, the equivalent program is command.com or cmd.exe, typically run in a console window. When you enter a command or statement shown in an example, do not type the prompt shown in the example. Database, table, and column names must often be substituted into statements. To indicate that such substitution is necessary, this manual uses db_name, tbl_name, and col_name. For example, you might see a statement like this: mysql> SELECT col_name FROM db_name.tbl_name; This means that if you were to enter a similar statement, you would supply your own database, table, and column names, perhaps like this: mysql> SELECT author_name FROM biblio_db.author_list; SQL keywords are not case sensitive and may be written in any lettercase. This training guide uses uppercase. In syntax descriptions, square brackets ('[' and ']') indicate optional words or clauses. For example, in the following statement, IF EXISTS is optional: DROP TABLE [IF EXISTS] tbl_name When a syntax element consists of a number of alternatives, the alternatives are separated by vertical bars ('|'). When one member from a set of choices may be chosen, the alternatives are listed within square brackets ('[' and ']'): TRIM([[BOTH | LEADING | TRAILING] [remstr] FROM] str) When one member from a set of choices must be chosen, the alternatives are listed within braces ('{' and '}'): {DESCRIBE | DESC} tbl_name [col_name | wild] An ellipsis (...) indicates the omission of a section of a statement, typically to provide a shorter version of more complex syntax. For example, INSERT ... SELECT is shorthand for the form of INSERT statement that is followed by a SELECT statement. An ellipsis can also indicate that the preceding syntax element of a statement may be repeated. In the following example, multiple reset_option values may be given, with each of those after the first preceded by commas: RESET reset_option [,reset_option] ... Commands for setting shell variables are shown using Bourne shell syntax. For example, the sequence to set the CC environment variable and run the configure command looks like this in Bourne shell syntax: shell> CC=gcc ./configure If you are using csh or tcsh, you must issue commands somewhat differently: shell> setenv CC gcc and shell> ./configure • •

s

o

an r t n

uy g N

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

e

bl a r fe

Supporting Images Used in This Training Guide The following is a summary of the standard images used in this manual to support the instruction:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

IMAGE

Thi

NAME

DESCRIPTION

Preparation

This image is used to describe the steps required to be completed prior to performing a hands-on exercise.

Written Exam

This image is used to identify that the student is going to be tested upon the material previously presented in the instructional material.

e

uy g N

bl a r Throughout the course the instructorfe InLine Lab ns will conduct labs in line with the a r t instruction, which are designed - to n o help you to understand the n of“nuts a and bolts” (inner-workings) the s topic. ha n) ideฺ v ฺ image to convey to Further Practice u isthatused om This G c thetstudent there is a final Lab ฺ n to complete prior to the 21 deexercise s t sp s Stu completion of the chapter. @ i n h a t r t This image identifies an area on a Student Notes e ฺ s n u o page designated for students to o o t m write notes associated with the ( nse n class. a r e c T i l et 123

Slide Number Boxes

This image is used throughout the course guide to indicate the existence and number of a corresponding instruction slide.

COURSE OBJECTIVES This course is designed for experienced database developers who wish to learn additional skills required to create complex queries and efficient structures while improving the performance of the database by learning to create and write queries that are optimized based on the data in the database.

5

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

At the completion of this course, you should be able to:

Thi

•

Utilize the multiple indexing options available to create better response times of query executions

•

Optimize queries that are responsible for searching text fields

•

Optimize queries that search date fields

•

Optimize queries that perform calculations and aggregations

•

Create summary tables that utilize triggers, event scheduler and table merges

•

Optimize queries that retrieves data from more than one table

•

Create tables that utilize tree and hierarchical structures

e

6

o n a • Utilize temporary tables to improve query performance s a h • Explain the affect locking levels have on query performance n) ideฺ v ฺ u ompreciset G • Create report queries that provide the end userscwith and organized data ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

an r t n

uy g N

s

bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Table of Contents 1 1.1 1.2 1.2.1 1.3 1.3.1 1.3.2 1.3.3 1.3.4 1.3.5 1.4 1.4.1 1.4.2 1.4.3 1.4.4 1.5 1.5.1 1.5.2 1.5.3 1.5.4 1.6 1.7 1.8 1.9 1.9.1 1.9.2 1.10 1.11 1.12

Introduction ................................................................................................................................................. 1-1 Learning Objectives .................................................................................................................................... 1-1 The MySQL Overview................................................................................................................................ 1-2 MySQL Partners.......................................................................................................................................... 1-3 MySQL Products......................................................................................................................................... 1-4 MySQL Database Products ......................................................................................................................... 1-4 MySQL GUI Tools...................................................................................................................................... 1-5 Other MySQL Tools.................................................................................................................................... 1-5 MySQL Drivers........................................................................................................................................... 1-6 Solutions for Embedding MySQL............................................................................................................... 1-6 MySQL Services ......................................................................................................................................... 1-7 MySQL Training ......................................................................................................................................... 1-7 MySQL Certification................................................................................................................................... 1-7 MySQL Consulting ..................................................................................................................................... 1-7 MySQL Support .......................................................................................................................................... 1-7 The MySQL Enterprise Subscription .......................................................................................................... 1-8 MySQL Enterprise Server ........................................................................................................................... 1-8 24x7 Enterprise Support.............................................................................................................................. 1-8 MySQL Enterprise Monitor ........................................................................................................................ 1-9 Obtaining a MySQL Enterpise Subscription ..............................................................................................1-12 Supported Operating Systems ....................................................................................................................1-13 MySQL Certification Program ...................................................................................................................1-14 Training Curriculum Paths .........................................................................................................................1-15 MySQL Website.........................................................................................................................................1-19 MySQL Community Web Page..................................................................................................................1-20 MySQL Online Documentation .................................................................................................................1-22 Installing MySQL.......................................................................................................................................1-25 Installing the 'world' database.....................................................................................................................1-26 Chapter Summary.......................................................................................................................................1-27

2.4 2.4.1 2.5 2.6 2.7

When MySQL Uses an Index...................................................................................................................... 2-8 When MySQL Does Not Use an Index ....................................................................................................... 2-9 Retrieving Index Information .....................................................................................................................2-11 Optimize Indexes........................................................................................................................................2-16 Summary ....................................................................................................................................................2-19

3 3.1 3.2 3.3 3.3.1 3.4 3.5 3.6 3.7 3.8 3.8.1 3.9 3.10

Searching Date and Text Fields................................................................................................................... 3-1 Overview ..................................................................................................................................................... 3-1 Strings in Multiple Columns ....................................................................................................................... 3-2 Case Sensitivity ........................................................................................................................................... 3-4 Binary searches ........................................................................................................................................... 3-7 INET_ATON and INET_NTOA................................................................................................................3-11 Searching Dates..........................................................................................................................................3-13 String Dates to SQL Dates .........................................................................................................................3-16 Dates to Integer Values ..............................................................................................................................3-18 Dates to Strings ..........................................................................................................................................3-23 Time values to Strings................................................................................................................................3-24 Nonspecific Date Searches.........................................................................................................................3-28 Summary ....................................................................................................................................................3-30

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nwith se Indexes ......................................................................................................... 2-1 2 Improving Performance n a r e 2.1 Overview 2-1 lic t T ..................................................................................................................................................... 2.2 yeQuery Executions in MySQL ...................................................................................................................... 2-2 2.2.1 gu Query Optimizer.......................................................................................................................................... 2-3 N hi 2.3 Why Indexes?.............................................................................................................................................. 2-4

T

e

bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

4 4.1 4.2 4.2.1 4.2.2 4.2.3 4.2.4 4.3 4.3.1 4.3.2 4.4 4.5

Improving Inserts ........................................................................................................................................ 4-1 Overview ..................................................................................................................................................... 4-1 INSERT Process.......................................................................................................................................... 4-2 Multiple Row Inserts ................................................................................................................................... 4-3 INSERT ... SELECT ................................................................................................................................... 4-4 LOAD DATA INFILE ................................................................................................................................ 4-4 Transactional Multiple Inserts..................................................................................................................... 4-8 Storage Engine Specifics............................................................................................................................. 4-9 Speeding up Bulk Inserts with MyISAM .................................................................................................... 4-9 Speeding up Bulk Inserts with InnoDB......................................................................................................4-10 MySQL Extensions ....................................................................................................................................4-11 Summary ....................................................................................................................................................4-18

5 5.1 5.2 5.3 5.4 5.5 5.5.1 5.5.2 5.6 5.7 5.8 5.9

Complex Calculations ................................................................................................................................. 5-1 Overview ..................................................................................................................................................... 5-1 Aggregate Multiplication ............................................................................................................................ 5-2 Running Total Query................................................................................................................................... 5-5 Avoiding Divide by Zero ............................................................................................................................ 5-7 Calculate the Median Value ........................................................................................................................ 5-9 Median Value via Implicit Temporary Tables ...........................................................................................5-10 Creating a Median Stored Procedure..........................................................................................................5-13 Simulating RANK ......................................................................................................................................5-16 Divide and Conquer ...................................................................................................................................5-18 Miscellaneous.............................................................................................................................................5-26 Summary ....................................................................................................................................................5-27

6.5.3 6.5.4 6.5.5 6.6

Double Checking EXPLAIN......................................................................................................................6-26 JOIN - Best Practices .................................................................................................................................6-33 Simulating INTERSECT and MINUS .......................................................................................................6-34 Summary ....................................................................................................................................................6-35

7. 7.1. 7.2. 7.3. 7.3.1. 7.3.2. 7.4. 7.4.1. 7.4.2. 7.4.3. 7.4.4. 7.4.5.

Hierarchical Data......................................................................................................................................... 7-1 Overview ..................................................................................................................................................... 7-1 Graphs, Trees and Hierarchies .................................................................................................................... 7-2 Adjacency List Structures ........................................................................................................................... 7-5 Search Tree Depths ..................................................................................................................................... 7-9 Modify Tree Values ...................................................................................................................................7-15 Nested Set Structures .................................................................................................................................7-16 Search Tree Depths ....................................................................................................................................7-18 Display Depth.............................................................................................................................................7-19 Displaying a specific branch ......................................................................................................................7-20 Inserting Nodes ..........................................................................................................................................7-22 Deleting Nodes...........................................................................................................................................7-25

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c 6 Improving Performance of Joins ................................................................................................................. 6-1 ฺ 1 n 2 e 6.1 Overview ..................................................................................................................................................... 6-1 s pt Stud 6.2 Query Optimizer.......................................................................................................................................... 6-2 s @ this 6.2.1 Greedy Optimizer........................................................................................................................................ 6-4 n a r 6.3 EXPLAIN.................................................................................................................................................... 6-6 t e ฺ s n 6.4 JOIN's.......................................................................................................................................................... 6-8 o to u o 6.4.1 Nested JOIN's.............................................................................................................................................6-14 m 6.4.2 Self Joins ....................................................................................................................................................6-15 n ( ense a r 6.4.3 FULL ic lJOIN..................................................................................................................................6-16 t T OUTER 6.5 yeImproving JOIN performance ....................................................................................................................6-21 u Reducing Rows Touched by Joins .............................................................................................................6-22 6.5.1 g N hi 6.5.2 Covering the Index .....................................................................................................................................6-24

T

e

bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

7.5. 7.5.1. 7.5.2. 7.6.

Path Enumeration .......................................................................................................................................7-28 Search the depths of the path......................................................................................................................7-29 Retrieving hierarchical data from the paths................................................................................................7-31 Summary ....................................................................................................................................................7-32

8. 8.1. 8.2. 8.3. 8.3.1. 8.4. 8.4.1. 8.5. 8.6.

Advanced Index Structures ......................................................................................................................... 8-1 Overview ..................................................................................................................................................... 8-1 MySQL Index Types ................................................................................................................................... 8-2 FULLTEXT Index....................................................................................................................................... 8-5 FULLTEXT Search Functions .................................................................................................................... 8-6 Simulating Function Based Indexes ............................................................................................................ 8-8 Searching for valid words............................................................................................................................ 8-8 Optimizing End of Field Searches..............................................................................................................8-10 Summary ....................................................................................................................................................8-14

9 9.1 9.2 9.3 9.3.1 9.3.2 9.4 9.4.1 9.4.2 9.4.3 9.5 9.6

Locking Levels............................................................................................................................................ 9-1 Overview ..................................................................................................................................................... 9-1 Locks ........................................................................................................................................................... 9-2 Explicit Table Locks ................................................................................................................................... 9-4 LOCK TABLES statement.......................................................................................................................... 9-4 UNLOCK TABLE statements..................................................................................................................... 9-5 Storage Engine Locking Techniques........................................................................................................... 9-6 MyISAM Locking ....................................................................................................................................... 9-6 InnoDB Locking.......................................................................................................................................... 9-9 Locking with Other Storage Engines .........................................................................................................9-10 Locking Issues............................................................................................................................................9-11 Summary ....................................................................................................................................................9-17

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t p Stu 10. Creating Reports.........................................................................................................................................10-1 s @ 10.1. Overview ....................................................................................................................................................10-1 is n h a t r 10.2. Calculate Multiple Conditions....................................................................................................................10-2 ฺt use n 10.3. Create a Calculated Report.........................................................................................................................10-4 o o to 10.4. Quarterly Reports .......................................................................................................................................10-9 m ( e 10.4.1. Cross Tab n Reports ....................................................................................................................................10-10 ns aChart..........................................................................................................................................10-11 r e 10.5. SQLTBar c t Tablesli........................................................................................................................................10-13 10.6. yeDecision u Materialized Views ..................................................................................................................................10-16 10.7. g N hi 10.8. Producing Sequential or Missing Data .....................................................................................................10-17

T

10.8.1. Using the integers table to fill in missing data .........................................................................................10-21 10.9. Summary ..................................................................................................................................................10-23 11 11.1 11.2 11.3 11.4 11.5

Conclusion..................................................................................................................................................11-1 Course Objectives ......................................................................................................................................11-1 Training and Certification Website ...........................................................................................................11-2 Course Evaluation ......................................................................................................................................11-4 THANK YOU!...........................................................................................................................................11-5 Q&A Session..............................................................................................................................................11-6

Appendix A Appendix B Appendix C Appendix D Appendix E

e

bl a r fe

Installation Procedures..................................................................................................................A-1 MyISAM Locking.........................................................................................................................B-1 Quiz Solutions...............................................................................................................................C-1 Further Practice Solutions .............................................................................................................D-1 Better Performance, Scalability, and Recovery.............................................................................E-1

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

CHAPTER 1 INTRODUCTION

e

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

1 1.1

Chapter 1: Introduction

INTRODUCTION Learning Objectives

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

This chapter introduces you to MySQL products and services. In this chapter, you will learn: 8

Thi

•

Explain the origin and status of the MySQL product

•

List the available MySQL products and professional services

•

Describe the MySQL Enterprise subscription

•

List the currently supported operating systems

•

Describe the MySQL Community web page

•

Describe the MySQL Certification program

•

List all the available MySQL courses

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-1

MySQL Developer Techniques

1.2

The MySQL Overview MySQL is a Relational Database Management System (RDBMS) that was originally developed by ‘MySQL AB’. It was (and continues to be) developed and marketed as a family of high performance, affordable database servers and tools. Contributing to building the missioncritical, high-volume systems and products worldwide is what makes MySQL the world’s most popular open source database, as well as its reliability, excellent performance and ease of use.

9

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 1: Introduction

MySQL is not only the world’s most popular open source database. It is also the fastest growing database in the industry, with over 70,000 downloads per day, ranging from large corporations to specialized embedded applications.

MySQL is installed on every ble continent in the world fera s (Yes, even Antarctica!) tran

MySQL AB was founded in Sweden by two Swedes and a Finn: David Axmark, Allan Larsson and Michael "Monty" Widenius who have worked together since the 80's. MySQL AB (Swedish for “Inc.”) was the sole owner of the MySQL server source code, the MySQL trademark and the mysql.com domain worldwide.

n o n

10

a s a h ฺ ) n ฺv uide m o Sun Acquisition: ฺc nt G 1 2 In early 2008, MySQL AB was acquired by Sun de Inc., giving the MySQL product line the tsMicrosystems, u p t s considerable resources of a major mainstream company.S Together, Sun and MySQL will now provide global @ s i enterprise-class support 24x7x365. Our enterprise-class customers worldwide can now take advantage of our n h t tra on their market-leading open source database choice of hardware, operating system and language with up to a e ฺ s n o to u 90% lower total cost of ownership than many traditional o m database solutions. ( nse n a r ce T i MySQL has always been, and will continue to be, a strong l t e supporter of the open source philosophy and the open source y gu community, and strives to work with partners who share the

Thi

N

same values and mindset. This will be no different within Sun. Sun is a world-leader in open source, with products and technologies such as Java, OpenSolaris, Open Office, NetBeans, OpenSparc and many more. The MySQL mission is still the same... Make superior database software available and affordable to all!

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-2

MySQL Developer Techniques

1.2.1

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

11

Chapter 1: Introduction

MySQL Partners MySQL has had the privilege of forming alliances with excellent partners and attracting some of the most impressive customers in the industry! When you join our ranks, you are joining a winning team with a wide variety of MySQL implementations. It never ceases to amaze us, the innovative and powerful ways in which our tools are being used. To name only a few;

Web / Web 2.0

e

OEM / ISV's

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ On Demand, SaaS, Hosting Telecommunications Enterprise 2.0 21 den s t u p t s sS @ i n h a t r t e Open-source onฺ to usis powering the World! o (m nse n a r ce T i l t "We have We went from experimental to mission-critical in a couple of uyeused MySQL far more than anyone expected.months." g Jeremy Zawodny--MySQL Database Expert Yahoo! Finance hi N

T

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-3

MySQL Developer Techniques

1.3

MySQL Products

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

1.3.1

12

Thi

Chapter 1: Introduction

MySQL Database Products

Sun provides MySQL database products to meet the needs of ISV/OEM, Enterprise, and Community Server users. MySQL database products are recognized for superior ease of use, performance, and reliability. • MySQL Enterprise Server o

The most reliable, secure and up-to-date version of the world's most popular open source database for cost-effectively delivering E-commerce, Online Transaction Processing (OLTP), and multiterabyte Data Warehousing applications. (Available only with the MySQL Enterprise subscription).

• MySQL Community Server o

The MySQL database server for open source developers and technology enthusiasts who want to get started with MySQL. Supported by the large MySQL open source community. Under the General Public License (GPL), benefits to the open source community include a commercialgrade framework that is free of charge.

e

bl a r fe

ns a r t - a reliable o The most popular choice for OEMs/ISVs who want to cost-effectively embed orn bundle o and high-performance relational database. an s MySQL Cluster a h ) o A fault tolerant database clustering architecture for deploying highly n ideฺavailable mission-critical v ฺ database applications. om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a lice t Tr

• MySQL Embedded Database

•

e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-4

MySQL Developer Techniques

1.3.2 13

Chapter 1: Introduction

MySQL GUI Tools The MySQL GUI Tools form a comprehensive graphical user interface to your MySQL database. These easy to use graphical tools enable database Developers and Database Administrators (DBAs) to be more productive.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

• MySQL Migration Toolkit o

Using a wizard-driven interface, the MySQL Migration Toolkit implements a proven methodology and walks you through the necessary steps to successfully complete a database migration project.

• MySQL Administrator o A powerful graphical administration console that enables you to easily administer your MySQL environment and gain significantly better visibility into how your databases are operating. • MySQL Query Browser

s

Bundled Tools

an r t n

e

bl a r fe

o An extremely user-friendly graphical tool for creating, executing, and optimizing SQL queries for your MySQL Database Server.

no a s a h n) ideฺ v 1.3.3 Other MySQL Tools ฺ m Gu o c ฺ t server: 14 MySQL has also developed additional tools to assist in the MySQL 1 use of the n 2 e s pt Stud • MySQL Workbench s @ database is design tool developed by MySQL. It is o MySQL Workbench a is n a visual h t r the highly anticipated of the DBDesigner4 project. MySQL ฺt successor seapplication n u o Workbench is currently available for Microsoft Windows in a GPL licensed open o o t m source edition as well as a low-priced standard edition which includes additional ( nse n time saving features for professionals. a r ce T i l t • MySQL Proxy uye o MySQL Proxy is a simple program that sits between your g N client and MySQL server(s) that can monitor, analyze or Thi transform their communication. Its flexibility allows for

The MySQL graphical user interface (GUI) tools discussed here come bundled together when downloaded from the MySQL website.

unlimited uses; common ones include: load balancing; failover; query analysis; query filtering and modification; and many more.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-5

MySQL Developer Techniques

1.3.4 15

Chapter 1: Introduction

MySQL Drivers MySQL Connectors are database drivers, providing database client connectivity for a wide range of programming languages. MySQL provides the following connectors:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

• MySQL C API o Our native client library (libmysql), which can be wrapped by other languages. • MySQL Connector/ODBC o Connect to a MySQL database server using the Open Database Connectivity (ODBC) API on all Microsoft Windows and most Unix-like platforms. The ODBC driver builds on the client/server protocol implementation provided by libmysql. • MySQL Connector/J o A JDBC (Java Database Connectivity) 4.0 driver for Java 1.4 and higher. Provides a java native implementation of the MySQL client/server protocol. • MySQL Connector/Net

s

an r t n

o A fully managed ADO.NET provider for the .NET framework (version 1.1 and 2.0). Provides a .NET implementation of the MySQL client/server protocol.

no a sare two MySQL specific PHP a o Provides MySQL connectivity for PHP programs. Currently there h nand) mysqli extensions available that use libmysql: the mysql eฺextensions. There is also v ฺ d i MySQL support for the generic PHP Databaseo objects m (PDO) uextension. In addition there is the G creplace ฺ PHP native driver called mysqlnd which1can libmysql in the mysqli extension. t n 2 e s 1.3.5 Solutions for Embedding MySQL pt Stud s @clienttside isconnectivity, MySQL also provides libraries to embed a Apart from these Connectors that provide n h a r MySQL database server within ฺatprogram. Currently se the following solutions can be used to embed MySQL: n u o • libmysqldmo to 16 ( e n embedded nsedition of the mysqld server program wrapped in a shared library. Allows the oaThe r e c T MySQL li Server to be embedded in C programs. t e y gu • MySQL MX/J N o A jar wrapper around mysqld binaries. This allows java programs and J2EE environments to Thi • MySQL Connector/PHP

e

bl a r fe

instantiate (and install) a MySQL server.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-6

MySQL Developer Techniques

1.4

MySQL Services

1.4.1

MySQL Training Sun Microsystems offers a comprehensive set of MySQL training courses that give you a competitive edge in building world-class database solutions.

17

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 1: Introduction

• Courses can be chosen individually, as part of a bundle, and/or following our suggested curriculum path for Developers and Database Administrators (DBAs).

1.4.2

MySQL Certification MySQL Certification Program is a high quality certification program that provides Developers and DBAs with the credentials to prove they have the knowledge, experience and skills to use and manage MySQL Server. With MySQL personal certifications, you get to show that you are among the best-of-breed MySQL users and lay the foundation for becoming a trusted and valuable resource for your company and customers.

1.4.3

bl We offer a full range of consulting services. Whether you are starting a new project, needing to optimize e anra sf existing MySQL application, or migrating from a proprietary database to MySQL, we have an affordable n a solution for you. -tr n o Using industry best practices and proven methodologies, your MySQL certified consultant nwill help you deliver a on-time and on-budget. s a h MySQL Support n) ideฺ v ฺ We also offer a full range of support options. MySQL Technical Support isu designed to save you time and to om G c ฺ ensure you achieve the highest levels of performance, reliability, and uptime. t 21 den s • Community support t sp s Stu o Mailing Lists @ i n h a t r o Forums (http://forums.mysql.com/) t e nฺ (http://dev.mysql.com/tech-resources/articles) us oArticles o Community o o t (m e o Bugs Databases (http://bugs.mysql.com/) n n a r e T o No direct lic access to support engineers t e o PlanetMySQL Blogs (http://www.planetmysql.org/) uy

1.4.4

g

N Thi

e

MySQL Consulting

o MySQL Reference Manuals (http://dev.mysql.com/doc) o MySQLForge (http://forge.mysql.com/)

• Purchased support o Enterprise subscription o Support for MySQL Cluster o Support for MySQL Embedded (OEM/ISV) o Online Knowledge Base

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-7

MySQL Developer Techniques

1.5

The MySQL Enterprise Subscription A MySQL Enterprise subscription gives you access to the Enterprise Server as well as other premium products and services. It is a comprehensive set of enterprise-grade software, support and services to ensure the highest levels of reliability, security and uptime. As a proactive service that helps you eliminate problems before they occur, it gives you everything you need in a single, unified offering to successfully develop and deploy business critical applications using MySQL.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

18

1.5.1 19

Chapter 1: Introduction

MySQL Enterprise Server The most reliable, secure and up-to-date version of the world's most popular open source database for costeffectively delivering E-commerce, Online Transaction Processing (OLTP), and multi-terabyte Data Warehousing applications. With an Enterprise subscription you also get the following in addition to the Enterprise Server: • Updates and Service Packs o

• Emergency Hot Fix Builds

s

an r t n

no a s a • Drivers h ฺ details. n) website efor o Many different drivers are supported. See MySQL Enterprise v ฺ d i om t Gu • MySQL Workbench c ฺ 1 n design and maintenance, automates 2simplifies e o A graphical user interface (GUI)tthat database s d p tasks,Sand tu improves communication among DBA and time-consuming and error-prone s @ s developer teams. i n h a t r t e ฺ • Platforms uares supported. See MySQL Enterprise website for details. onplatforms o o Many different o t m se (Support n 1.5.2 24x7 Enterprise n a r e c T i l et• Service Level y 20 u o MySQL Enterprise gives you the flexibility to choose a service level that matches your g N i requirements. Th o

Can request a special hot fix build that resolves a critical issue that is not addressed by a service pack.

ƒ

Basic, Silver, Gold, or Platinum (with incrementally increasing quantity and quality of support)

• Problem Resolution Support o

Ensures you receive high priority service from the MySQL Support Team for quick resolution of technical problems as they occur. The services available to you are dependent on your chosen service level.

• Consultative Support o

e

bl a r fe

Automatically receive both monthly rapid updates and quarterly service packs with the latest bug-fixes and security updates.

Expert MySQL Consultants provide you with proactive advice on how to properly design and tune your MySQL servers, schema, queries and replication to ensure the highest possible reliability.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-8

MySQL Developer Techniques

Chapter 1: Introduction

• Technical Account Manager (TAM)

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

o

You have the option of adding a Technical Account Manager (Platinum level only) to be your liaison within MySQL. Your TAM be your single point of contact, providing a custom review of your systems, regular phone calls and on-site visits, to guarantee that you get the most out of MySQL Support Services.

• Online Knowledge Base o

For quick self-help knowledge, you will have access to a comprehensive and easily searchable knowledge base library with hundreds of technical articles regarding difficult problems on popular database topics such as performance, replication, configuration, and security. Enterprise support

For more detailed information regarding Enterprise support, please refer to our Enterprise web page located at: http://www.mysql.com/products/enterprise/support.html .

1.5.3 21

Thi

MySQL Enterprise Monitor

s

an r t n

The MySQL Enterprise Monitor is a web-based monitoring and advising system. The Enterprise Monitor helps MySQL DBAs manage more MySQL servers in a scale-out environment, tune their current MySQL servers and find and fix problems with their MySQL database applications before they can become serious problems or costly outages. Running completely within the corporate firewall, the Enterprise Monitor pro-actively monitors enterprise database environments and provides expert advice on how MySQL can tighten security, optimize performance and reduce downtime of their MySQL powered systems. EM accomplishes all this while reducing DBA time and effort. EM provides a rich GUI Enterprise Dashboard that contains “pages” which allow the DBA to have an immediate, graphic view of administrative tasks, server and database status, and advisory information. The principle features of Enterprise Monitor:

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e • Enterprise Dashboard onฺ o usfrom a consolidated console o Manage o all MySQLtservers (m nse management • Server or Server-group n a ro Autolidetection, ce grouping and monitoring of replication and scale-out topologies T t e

uy g N

e

bl a r fe

• Monitoring page o "At a glance" global health check of key systems

• MySQL provided Advisors and Advisor Rules o Enforce MySQL Best Practices • Advisor Rule Scheduler o Schedule unattended operations • Customizable Thresholds and Alerts o Identify Advisor Rule violations • Custom Advisor Rule o User defined advisor rules • Events and Alert history o Lists all Advisor Rules executions • Specialized Scale-Out assistance ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-9

MySQL Developer Techniques

Chapter 1: Introduction

Enterprise Dashboard

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

22

Graphs (custom)

Page Tabs

ED Control /Help

e

bl a r fe

s

an r t n

o

an s a h ) Explorer n ideฺ v Navigation ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i Critical Events l t e y gu N hi

Heat Chart

T

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-10

MySQL Developer Techniques

MySQL Enterprise Advisors The MySQL advisors are divided into the following categories (and specific functions):

23

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 1: Introduction

e

bl a r fe

s

an r t n

24

no a s a Enterprise Monitor Subscription Level h n)features eareฺ only available with the A “Silver” or higher level is required to get Enterprise Monitor. ฺSome v d i u below: “Gold” and “Platinum” levels, respectively. A list of features per level is shown om G c ฺ t 1 Gold n e Silvers2 Platinum d t u p t s S Enterprise Dashboard @ s i n h tra se t ฺ Notifications andn Alerts oo to u m Custom ( Advisor n ense a r TUpgrade Advisor lic t e guy Administration Advisor

Thi

N

Security Advisor Replication Monitor Replication Advisor Query Analysis Advisor Memory Usage Advisor Schema Advisor Performance Advisor ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-11

MySQL Developer Techniques

1.5.4

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

25

Thi

Chapter 1: Introduction

Obtaining a MySQL Enterpise Subscription For volume discounts or to order by phone, the MySQL Enterprise subscription can be obtained by contacting Sun Microsystems/MySQL sales personnel:

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

A permanent MySQL Enterprise subscription can be purchased through the MySQL website located at : https://shop.mysql.com/enterprise/. A 30-day trial (with limited features) is also available: http://www.mysql.com/trials/ .

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-12

MySQL Developer Techniques

1.6

Supported Operating Systems

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

26

Thi

Chapter 1: Introduction

MySQL runs on more than 20 platforms, giving users the kind of flexibility that puts them in control. Windows, Linux and Solaris are the most popular operating systems for running MySQL. Versions of MySQL are currently available for the following operating systems: • FreeBSD • HP-UX • IBM AIX and i5 • Linux (multiple) • Mac OS/X • Microsoft Windows (multiple) • Novell netware

e

• Open BSD • QNX • SGI Irix

bl a r fe

s

an r t n

no a s • Source code a h • Special builds n) ideฺ v ฺ m Gu oinformation, c This list is continually being updated. For the most current ฺ t please check the MySQL website at: 1 n 2 e http://www.mysql.com . s pt Stud s @ this n a r ฺt use n o o to m ( e an icens r T l t e y gu • Solaris

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-13

MySQL Developer Techniques

1.7

MySQL Certification Program The MySQL Certification Program is a high quality certification program that provides developers and DBAs with the credentials to prove they have the knowledge, experience and skills to use and manage MySQL Server. MySQL provides several certification types and levels:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

27

Thi

Chapter 1: Introduction

1. Certified MySQL Associate (CMA) - an entry level certification. It is intended for those that are relatively new to using the MySQL database server and covers basic database management system concepts as well as basic SQL. We recommend the CMA certification for MySQL users that know the basics, but have not yet obtained the experience gained by professional MySQL DBAs or Developers. 2. Certified MySQL 5.0 Developer (CMDEV) - proves mastery of the fundamental skills of using MySQL including creating and using databases and tables, inserting, modifying, deleting, and retrieving data. 3. Certified MySQL 5.0 Database Administrator (CMDBA) - proves mastery of the ability to manage MySQL Server including such advanced areas of database management, installation, security, disaster prevention and optimization.

s

an r t n

e

bl a r fe

4. Certified MySQL Cluster DBA (CMCDBA) - part of the DBA track which represents an advancement level exceeding CMDBA certification. In order to attain CMCDBA certification, you must attain CMDBA certification and pass one CMCDBA exam.

no a s a h Certification web page nthe) exams, eseeฺ our Certification web v ฺ d For more information on the certification program and the content of i om t Gu page; http://www.mysql.com/certification/ . c ฺ 21 den Exam administration s t tuPearson VUE testing centers available worldsp s 3,000 S All exams are administered through one@ of more than i proctor network. Visit the MySQL certification n h wide. And through the Linux r Professional Institute a t t e webpage, online forum or email for more information. s u onฺ[email protected] o o (m nse t n a ice r T l t e y gu

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-14

MySQL Developer Techniques

1.8

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

28

Thi

Chapter 1: Introduction

Training Curriculum Paths MySQL offers the most comprehensive set of MySQL training courses that enable the building of database solutions and competitiveness now and into the future. In addition to our open courses, we also offer in-house training. The MySQL training services staff has put together great courses designed for success, and an excellent training path for each individual to reach his/her training goals. There are two tiers to the curriculum paths: Developer path •

Introduction courses o

o

s

an r t n

e

bl a r fe

MySQL for Beginners – This course covers the fundamentals of SQL and relational databases, using MySQL as a teaching tool. Prerequisites: Basic computer literacy is required. Previous experience with any command-line program (such as MS-DOS or the MS Windows command prompt) is beneficial. 4 Days in length.

no a s a h • Intermediate courses n) ideฺ v ฺ m is designed u for students planning on developing o MySQL for Developers - This instructor-led o course G c ฺ t applications that make use of MySQL 5.0 (and higher). This course covers essential SQL statements 1 In addition, n 2 e for data design, querying, and programming. it will prepare students for the MySQL s d with Relational Databases u pt Some t Developer certification. Prerequisites: experience and SQL. 5 days s S @ s i in length. an e th r t ฺ s New Features – This instructor-led course will provide in-depth nUpgradinguand o MySQL 5.0 oneeded o o knowledge to become proficient using MySQL 5.0. This training course will provide quality t (m time both on s theetopic, hands-on labs and with the expert instructor. Prerequisites: Existing MySQL n n a usersiwho r e want to become proficient using MySQL 5.0. 3 Days in length. c T l t

e•

uy g N

MySQL and PHP : Developing Dynamic Web Application –This course will provide the tools needed for the development of dynamic web application. This course takes the student from the basics to the advanced features in four hands-on, heavy duty training days. Each student will have the opportunity to build a real-world application during the course. Prerequisites: Basic experience with designing HTML pages including HTML forms and experience with any programming language. Basic PHP skills. 4 days in length.

Advanced courses o

MySQL Stored Procedure Techniques – This instructor-led course with focus on labs is designed to teach you how to maximize the use of stored procedures along with the knowledge to discern when an application should contain stored procedures and when they should not. 2 Days in length.

o

MySQL Developer Techniques - This instructor-led course will provide database developers with additional skills in the design, development and maintenance of data and the queries to mine such data. This course places an emphasis on best practices for developers that improve response time of query executions. The database developer will also learn additional query writing techniques for creating reports, and creating trees and hierarchies, that can be used within MySQL to support the needs of the end users. Prerequisites: Having attended the MySQL for Developers course, or similar knowledge. 3 Days in length.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-15

MySQL Developer Techniques

Chapter 1: Introduction

Database Administrator (DBA) Path •

Introduction courses

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

o

Thi

•

•

MySQL for Beginners – This course covers the fundamentals of SQL and relational databases, using MySQL as a teaching tool. Prerequisites: Basic computer literacy is required. Previous experience with any command-line program (such as MS-DOS) is beneficial. 4 Days in length.

Intermediate courses o

MySQL 5.0 Upgrading and New Features – This instructor-led course will provide in-depth knowledge needed to become proficient using MySQL 5.0. This training course will provide quality time both on the topic, hands-on labs and with the expert instructor. Prerequisites: Existing MySQL users who want to become proficient using MySQL 5.0. 3 Days in length.

o

MySQL for Database Administrators – This course covers essential DBA tasks such as, installation and upgrading, user management, disaster recovery, and optimization. In addition, it will prepare students for the MySQL Database Administrator certification. Prerequisites: Some experience with Relational Databases and SQL. 5 days in length.

Advanced courses

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 1 n 2 e s d t o MySQL Cluster - Learn how to andtconfigure the cluster nodes to ensure high availability. u sp install S Also learn about all of the high availability features that are implemented in the new storage engine @ s i n— fail-over h for the MySQL Cluster between storage nodes, network partitioning protocol, two-phase a t r t e commit and much Prerequisites: Attendance to the MySQL for Database Administrators or an s nฺ more. u omastery o equivalent of database concepts, SQL and the MySQL server. 3 Days in length. o t m ( e nsis the sole course offering in the Cluster certification (CMCDBA) path. This an Note: r e c T li High Availability - This course is designed for experienced database administrators and et o MySQL o

uy g N

e

bl a r fe

MySQL Performance Tuning – The MySQL Performance Tuning course is designed for Database Administrators and others who wish to monitor and tune MySQL. This course will prepare each student with the skills needed to utilize tools for monitoring, evaluating and tuning. Students will evaluate the architecture, learn to use the tools, configure the database for performance, tune application and SQL code, tune the server, examine the storage engines, assess the application architecture, and learn general tuning concepts. 4 Days in Length.

system architects that want to analyze and form a basis of understanding different high availability options, including clustering and replication solutions within MySQL. This course will provide the tools required to make the decision of what high availability solution is appropriate and how to implement a system with the correct design.. 3 days in length.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-16

MySQL Developer Techniques

Chapter 1: Introduction

Introductory

MySQL and PHP

Thi

MySQL for Beginners

4 days

Advanced/ Intermediate Special Topics

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Curriculum Path Chart

Certification: CMA

4 days

MySQL for Developers

MySQL for DBAs

Certification: CMDEV I-II

Certification: CMDBA I-II

s

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t Developer DBA e ฺ s n u o o to m ( e an icens r T l t

MySQL Stored Procedure Techniques 2 days

an r t n

MySQL Developer Techniques

3 days

MySQL Performance Tuning

MySQL High Availability

4 days

3 days

MySQL Cluster 3 days Certification: CMCDBA

e

uy g N

e

bl a r fe

5 days

5 days

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-17

MySQL Developer Techniques

Chapter 1: Introduction

Virtual Classroom MySQL also offers virtual (online) courses covering various topics in relation to the MySQL suite of tools. These classes are instructor-led and delivered synchronously, via the web. Information regarding the available courses can be found on the MySQL training web page.

Introductory

PHP Elements

Thi

Coming Soon!

4 hours

MySQL Enterprise Monitor 4 hours

e

Advanced/ Intermediate Special Topics

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

29

MySQL Transactions

MySQL Storage Engines

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ n 21ComingdSoon! e s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r Developer DBA ce T i l t e 4 hours

Writing MySQL UDFs

2x4 hours

4 hours

4 hours

s

an r t n

MySQL Partitioning

bl a r fe

MySQL Proxy

4 hours

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-18

MySQL Developer Techniques

1.9

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

30

Thi

Chapter 1: Introduction

MySQL Website Everything you ever wanted to know about MySQL and more can be found on our website: http://www.mysql.com. From the home page, you may navigate the web site with the tabs (pull-down menus) across the top, the menu along the left side, and/or the many links on the page.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-19

MySQL Developer Techniques

1.9.1

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

31

Thi

Chapter 1: Introduction

MySQL Community Web Page The MySQL Community web page is located at http://dev.mysql.com and is maintained by Sun Microsystems, Inc. This is the main support tool for the MySQL open source community and can provide valuable insight for Enterprise users also. Information such as the following can be found:

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-20

MySQL Developer Techniques

Information such as the following can be found on the Developer Zone web page:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

32

Thi

Chapter 1: Introduction

• Current product and service promotions • Get Started with MySQL o Installation information page for MySQL beginners • Developing with: o Links to specific information on using MySQL with; PHP, Perl Python, Ruby, Java/JDBC and .Net/C#/Visual Basic • Quality Contribution Program o Structured program for software contributions and bug reports o Contributors acknowledged/rewarded • MySQL Server Community Edition o All current General Available (GA), Release Candidate (RC) and Alpha versions of the MySQL Server • New Releases o Latest code release for all MySQL products • Software Previews o New features available for preview by users

o

an What's New s a o Latest news about the company, products, services, etc. h ) n ideฺ v MySQL Training ฺ m Services u oTraining o List of upcoming courses provided by MySQL G c ฺ t MySQL Quickpoll 21 den s t tu issues o Give your opinions about current spMySQL-related S @ s i Stay Connected n h a t r t e o Blogs, Lists, Guilds, s Meetups and Quality Contributions nฺ MySQL u o o o Resources m ( nsArticles, e t Training, Webinars, and MySQL Newsletter o n White Papers, a more!ice r T And much l t

• • • • • •

s

an r t n

e

bl a r fe

e guy

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-21

MySQL Developer Techniques

1.9.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

33

Thi

Chapter 1: Introduction

MySQL Online Documentation Documentation for all MySQL products can be found from our website, on the “Documentation” page: http://dev.mysql.com/doc/ . The page includes links for downloading the following; • • • • • • • • • •

MySQL Reference Manual Excerpts from the Reference Manual MySQL GUI Tools Manuals Expert Guides MySQL Help Tables Example Databases Meta Documentation Community Contributed Documentation Printed Books Additional Resources

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-22

MySQL Developer Techniques

Chapter 1: Introduction

Inline ne Lab 1-A

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this exercise you will review some web pages on the MySQL website using the web browser of your choice.

Thi

Step 1. Place cursor in web browser URL field. Click in the text area of the Address/Location Toolbar. The URL currently listed will be selected. Step 2. MySQL web address In the toolbar text area, type: www.mysql.com

e

bl a r fe

The MySQL Homepage is displayed. Step 3. Products

s

an r t n

no a sand downloads. a A list of currently available products will appear, with links for further information h n) ideฺ v ฺ Step 4. MySQL Enterprise om t Gu c ฺ non the Learn More >> link. Review the details of the new MySQL Enterprise program, 21 by clicking e s d t p set ofSenterprise-grade tu MySQL® Enterprise™ provides a comprehensive software, support and services s @ s directly from the developers of MySQL to ensure the highest levels of reliability, security and uptime. i n h a t r t e onฺ to us Step 5. Services o (mtab located senear the top of the MySQL home page. Click on the Services n n a r e A listtofT currently available lic services will appear, with links for further information and downloads. e y guStep 6. MySQL Training and Certification

Click on the Products tab located at the top of the MySQL home page. Scroll down the list to review the various product information provided.

N

Review the details of the MySQL Training & Certification program, by clicking on the Learn More >> link. Featured information on this page will be updated periodically. For specific Training sub-topics select one of the links in the sub-menu in the upper-left corner of the page. Step 7. Certification From the MySQL Training & Certification web page, select the Certification link in the sub-menu and review the contents. Featured information on this page will be updated periodically. For specific Certification sub-topics select one of the links in the sub-menu in the upper-left corner of the page. ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-23

MySQL Developer Techniques

Chapter 1: Introduction

Step 8. Return to Services Click on the Services tab located at the top of the MySQL home page.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Returns to the top level Services page.

Thi

Step 9. Support Services Review the details of MySQL Support programs, by clicking on the Learn More >> link. Shows the various support programs available. Step 10. MySQL Community Click on the Developer Zone tab located at the top of the MySQL home page. The community page contains many links to more information and access to various open communication forums.

ns a r t Return to the MySQL home page using the MySQL.com tab at the top of the page, then click on- the News on & Events tab located near the top of the MySQL home page. n a Shows the latest MySQL happenings, from news stories to upcoming seminars. s a h ฺ ) n ฺv uide m o ฺc nt G 1 2 e pts Stud s @ this n a r ฺt use n o o to m ( e an icens r T l t e y gu Step 11. News & Events

e

bl a r fe

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-24

MySQL Developer Techniques

Chapter 1: Introduction

1.10 Installing MySQL

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

34

Thi

It is recommended to download the MySQL database server from the MySQL web site downloads page: http://dev.mysql.com/downloads/. There are several different platforms supported, and there are differences in the specific installation details for each. Inline Lab 1-B This lab requires you to use the Windows OS Wizard for installing the MySQL server. Step 1. Install MySQL For installations on the Windows and Linux Operating systems, see Appendix A at the back of this training guide. See Appendix A.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-25

MySQL Developer Techniques

Chapter 1: Introduction

1.11 Installing the 'world' database

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

35

Thi

MySQL provides three example databases to be used for testing server features and training. These databases can be found on the MySQL website documentation page; http://dev.mysql.com/doc/ . The ‘world’ database will be used throughout this course. Inline Lab 1-C This lab requires you to use the MySQL command line client in order to load the world database. In order to use the pre-made tables in the world database, you will need to create the database (empty) and then upload the file containing the table data. Step 1. Create 'world' database Type the following in the MySQL command line client: mysql> CREATE DATABASE world;

e

Creates the (empty) world database. Returns following message; Query OK, 1 row affected (#.## sec)

bl a r fe

s

an r t n

no a s Type: a h n) ideฺ v mysql> USE world; ฺ u om G Instructs client to use the newly created world database. Returns following message; c ฺ t 21 den s Database changed t sp s Stu @ i n h Step 3. Build 'world' database tables a t r t e onฺ to us Type:* o (m nse mysql> n SOURCE C:/world.sql a rwill giveliyou cethe file path if it is different than shown here. Several Query messages will scroll T * Instructor t ye while tables and data are being uploaded for the world database. gupassed Step 2. Select the 'world' database

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-26

MySQL Developer Techniques

Chapter 1: Introduction

1.12 Chapter Summary

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

36

Thi

This chapter introduced you to MySQL products and services. In this chapter, you learned: •

Explain the origin and status of the MySQL product

•

List the available MySQL products and professional services

•

Describe the MySQL Enterprise subscription

•

List the currently supported operating systems

•

Describe the MySQL Community web page

•

Describe the MySQL Certification program

•

List all the available MySQL courses

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

1-27

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 2

e

s

bl a r fe

an r t n

IMPROVING as a no h ฺ ) n PERFORMANCE ฺv WITH ide m u o ฺc nt G 1 2 INDEXES s de spt Stu

uy g N

@ this n a r ฺt use n o o to m ( e an icens r T l et

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

2.1

Thi

Chapter 2 - Improving Performance with Indexes

IMPROVING PERFORMANCE WITH INDEXES Overview This chapter provides the knowledge and skills necessary to improve the response time of queries by choosing and utilizing the multiple indexing options available. At the completion of this chapter, you will be able to:

38

•

Describe how MySQL handles query executions

•

Explain why MySQL uses indexes

•

List when MySQL uses and index and when it does not use an index

•

Describe ways to retrieve index information

•

Explain processes to optimize indexes

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-1

MySQL Developer Techniques

2.2

Query Executions in MySQL When a query is executed in MySQL, there are multiple layers of the process that must be considered.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

39

Thi

Chapter 2 - Improving Performance with Indexes

MySQL Database Management Level - within this level, the actual parsing of the the query statement is performed along with the optimization and execution of that statement. Storage Engine Implementation - within MySQL there are multiple storage engines that can be used to house the data that MySQL database management level will execute against. The storage engine itself will provide benefits (and limitations) based on their intended purpose and design.

MySQL Database Management Level

Optimize

Parse

Execute

Storage Engine Implementation MyISAM

InnoDB

MySQL Cluster

Falcon

Other ...

File System

Hardware

e

bl a r fe

s

File System - the file system that is being used plays a vital role in the overall performance of the query execution. There will be advantages (and disadvantages) for each file system and care must be taken to ensure that the file system that is chosen for MySQL to run on will improve the performance of query executions (and other vital database responsibilities) rather than provide a hindrance to performance.

an r t n

no a s a h ฺ and managing nfor)actually eexecuting v Hardware - last but not least, the hardware that is responsible ฺ d i m role inGtheuperformance and execution of the operating system and the installed software plays o a key c ฺ t queries against the MySQL database. 1 n 2 e s pt Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-2

MySQL Developer Techniques

2.2.1

Query Optimizer The task of the query optimizer in MySQL is to find the best or optimal plan for executing the SQL query that is submitted to the server. The query optimizer searches for the best plan among all the possible query evaluation plans. A process called greedy optimization enables the optimization engine to significantly reduce the amount of time it spends calculating optimal execution paths by a process of intelligent best-path reductions. For larger more complex queries, this became a bottleneck in the server’s performance. The following representation demonstrates the Query Process:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

40

Thi

Chapter 2 - Improving Performance with Indexes

MySQL DBMS Query Compiler Parser Analyze Syntax Preprocessor – Semantic checking, name resolution

Optimizer generate optimalQuery QueryExecution ExecutionPlans Plans(QEP’s) ( Generate Optimal

e

bl a r fe

ns

Parse-tra Tree non

a s a Query Transformations h ฺ ) n ฺv uide m o ฺcplan nt G Search for optimal execution 1 2 e pts Stud s @ is Plan refinement Query n h a t r t e Execution onฺ to us o Plan (m nse QEP n a ice r T l et

uy g N

Query execution engine

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-3

MySQL Developer Techniques

2.3

Why Indexes? There is a cost associated with a query execution needing to interact with a layer outside of itself in the form of time. When a storage engine has to be accessed, there is a time associated with reading the data. When a storage engine has to interact with the file system (such as cache misses on the file system), there is a time associated with that interaction. When the file system must interact with components of hardware that are slower than others (such as disk seeks versus being able to retrieve the data stored in memory) there is a loss in time to complete a query. With each of these there are steps that can be followed to improve performance, but for the most part the greatest gain in query execution performance involves indexes. Indexes Indexing increases the performance of the database when it comes to data-access performance by allowing quick access to the rows in a table. In addition, indexes can have positive effects on UPDATE and DELETE operations by providing a faster means to locate the rows affected. Indexes are created using one or more columns of a table. In addition to the index being smaller than the original table (due to having fewer columns), indexes are optimized for quick searching, usually via a balanced tree. When indexes are not used or are not matched by SQL statements submitted to the database, then a full-table scan is executed. A fulltable scan will read all rows in a table to find a specific row or set of rows, which can be extremely inefficient.

e

bl a r fe

s

42

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e Full Table Scan vs. Utilizing Indexes

100,000

10,000

uy g N

Milliseconds

1,000 Full Table Scan Utilizing Indexes

100

10

65 ,5 36

32 ,7 68

8, 19 2 16 ,3 84

4, 09 6

2, 04 8

1, 02 4

51 2

25 6

12 8

64

32

16

8

4

2

1

1

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

41

Thi

Chapter 2 - Improving Performance with Indexes

Number of Rows

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-4

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

43

Thi

Chapter 2 - Improving Performance with Indexes

Index Issues The following list decribes some index issues to be aware of:

mysql>

•

Speed versus maintenance – Indexes help to speed up data retrieval but are expensive to maintain. As indexes are added, the time to write to the data is increased to maintain index integrity. ○ UNIQUE indexes – A UNIQUE index creates a constraint such that all values in the index must be distinct. An error occurs if a new row of data is added with a key that matches an existing row. If an index is unique, use the UNIQUE clause to force the column to be distinct which in turn also improves the efficiency of searching on that column.

•

Index selectivity - The more selective an index is, the more benefit is obtained from using it. The more unique values in an indexed field, the better chance that the index has of finding the value being searched in an effective amount of time. Index selectivity of a field can be determined by using the following SQL statement:

SELECT 1.0/NULLIF(COUNT(DISTINCT(index_field)),0) FROM table;

e

bl a r The NULLIF is placed into the SQL to ensure that there is no chance of a divide by zero error. sisfe n A response of 0.50 is the worst possible result and states that the effectiveness of thea index r extremely poor. A response as close to 0.00 as possible is the best resultnto-tensure the o effectiveness of the index. n areference s For large datasets, this SELECT statement can be expensive to run (in to the time it a h would take). Using the INFORMATION_SCHEMA database, the following SQL will list the n) ideฺ v percentage of rows that are not unique: ฺ om t Gu c ฺ mysql> SELECT TABLES.TABLE_SCHEMA, TABLES.TABLE_NAME, 21 den s -> STATISTICS.INDEX_NAME, t sp s Stu'%') AS `Rows per value` -> CONCAT((1 - CARDINALITY/TABLE_ROWS)*100, @ -> FROM information_schema.STATISTICS i n h a t -> JOIN information_schema.TABLES r t e -> USING (TABLE_SCHEMA, us onฺ ANDtoTABLE_NAME) o -> WHERE non_unique=1 TABLE_ROWS/CARDINALITY IS NOT NULL; (m nse n a - Highly duplicated data should not be indexed (for example, boolean data r ○ Cardinality ceand columns T i l t types, that represent gender, state abbreviations, or country codes). However, e y having a heavily skewed data distribution can make indexes useful when looking for some u g

N

values and not useful when looking for others.

Altering the execution of a query In a situation where the end user understands the data, MySQL provides two means to alter the execution of the query: • USE INDEX(index_list) - This clause will tell the MySQL server to only evalute the indexes in the index_list to determine which index (if any) would be the best to use for the query. If the MySQL server believes that a full table scan is the most appropriate execution to provide the best results, this clause makes that possibility available. • FORCE INDEX(index_list) - This clause is identical to the USE INDEX clause but will force MySQL to choose one of the indexes in the index_list. A full table scan would never be performed with this clause even if the MySQL server felt it would provide the best results. This clause would be appropriate in those times when the cardinality data type field indexed is searching for the more unique values in the field. ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-5

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

Chapter 2 - Improving Performance with Indexes

•

Short keys - Keys that are shorter in length have benefits over longer keys due to the sheer speed of eliminating the need to scan a long value. However, too short of a key reduces the possibility of having index selectivity.

•

Integer data types - Keys that are based on the integer data type make the best indexes not only for index operations but also for join and other types of database operations.

•

Dead Indexes - Make sure to avoid indexes that are never used by any queries. These cause additional overhead that is not necessary, and removing them will improve overall efficiency, especially during updates, deletes and inserts.

•

Duplicate indexes - Avoid more than one index on the same column(s). The optimizer must determine which to use. Also there is more maintenance as the data changes.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-6

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

44

Thi

Chapter 2 - Improving Performance with Indexes

Composite Indexes MySQL uses multiple-column indexes in such a way that queries are fast when a known quantity is specified for the first column of the index in a WHERE clause, even if values are not specified for the other columns. The following list of characteristics describe composite indexes: •

Indexes can be created on several columns.

•

They can be used for searches on just the first column(s).

• An index on (a,b,c) can be used for searches on [(a), (a,b), (a,b,c)]. In summary, a composite index spans multiple columns where each column is sorted based on the value in the preceeding column. For example, the following describes what a composite index on the Country table which includes the Continent, Region and Name columns would look like when it is stored: Continent Antarctica Antarctica ... Africa Africa Africa Africa Africa Africa ... Africa Asia Asia ... Asia Europe Europe Europe Europe Europe Europe ...

Region Antarctica Antarctica

Name Antarctica Bouvet Island

Central Africa Central Africa Central Africa Eastern Africa Eastern Africa Eastern Africa

Equatorial Guinea Gabon Sao Tome and Principe British Indian Ocean Territory Burundi Comoros

Eastern Europe

Belarus

e

s

an r t n

o

an s a h ) n ideฺ v ฺ Western Africa Togo om t Gu Eastern Asia China1ฺc 2 Kong den s Eastern Asia Hong t sp s Stu @ i Southern and Central n Asia thUzbekistan a r t Baltic Countries Estonia e nฺ us Latvia oCountries Baltic o o t Baltic Countries Lithuania (m e s n n British Islands Ireland a r e c T i British Islands United Kingdom l et

uy g N

bl a r fe

In this scenario, the Continent column is sorted and then the Region column is sorted within the Continent column and finally the Name is sorted within the Region column. This indexing method allows the MySQL server to quickly locate records when the first column is called along with the other columns in an ascending fashion. This type of index would not be useful if the third column was searched without searching the first and second columns. Leftmost index prefixes In a table that has a composite (multiple column) index, MySQL can use leftmost index prefixes of that index. A leftmost prefix of a composite index consists of one or more of the initial columns of the index. MySQL's capability to use leftmost index prefixes enables you to avoid creating unnecessary indexes. In the composite index example above, the index on column a along with the index on columns a and b are considered leftmost prefix indexes.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-7

MySQL Developer Techniques

2.4 45

When MySQL Uses an Index The following list entails a checklist of sorts that describes when the MySQL server will actually use an index:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

•

Thi

Chapter 2 - Improving Performance with Indexes

Testing an index against a constant value - When an index value is tested against a constant, the MySQL server will utilize the applicable index:

mysql > SELECT ... FROM Country WHERE Code = 'NOR';

•

Joining against an indexed column - When an index value is tested against a joined field, the MySQL server will utilize the applicable index:

mysql> SELECT ... FROM Country

•

JOIN City ON Capital = Id;

Leftmost index prefixes - When utilizing indexes, the MySQL server allows for the use of the leftmost part of a prefix index only. In the following example, the CountryLanguage table has a composite index on the CountryCode and Language columns. In the following example only the leftmost part of that composite index (the prefix index) is being utilized:

e

bl a r fe

ns a r t -= 'USA'; n mysql> SELECT Language, Percentage FROM CountryLanguage WHERE CountryCode o a nstatements. The only s • LIKE searches - Index values can be searched using wildcards with LIKE a h limitation to this method is the wildcard can not be the first character: ) n ideฺ v ฺ u mysql> SELECT Name, Code FROM Country WHERE Code LIKE 'Ma%'; om G c ฺ t n 21 serverdeprovides s • MAX() and MIN() searches - ThetMySQL for MAX() and MIN() functions p Stu s to utilize indexes as well: @ this n a r ฺt City;use mysql> SELECT MAX(Id) FROM n o oresults - tThe o MySQL server will utilize the index to produce a result set that has m ( • Presorting e n sortedewhen ns the column being sorted is an indexed field: abeen r c T li t e y mysql> SELECT Name FROM City ORDER BY Id LIMIT 5; gu

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-8

MySQL Developer Techniques

2.4.1 46

When MySQL Does Not Use an Index The following list entails a checklist of sorts that describes when the MySQL server will not use an index:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

•

Thi

Chapter 2 - Improving Performance with Indexes

Specific wildcard searches - As stated earlier, the MySQL server will not use an index when a wildcard search utilizes a LIKE statement that begins with a wildcard:

mysql> SELECT Name FROM Country WHERE Code LIKE '%M';

•

Later parts of a composite index - The MySQL server will not use an index if the first part of a composite index is not used. Thus in the following example, the first part of the composite index on the CountryLanguage table is not being used and thus the composite index made up of the CountryCode and Language columns will not be used:

e

bl a r fe

mysql> SELECT CountryCode, Language, Percentage FROM CountryLanguage > WHERE Language = 'French';

•

s

an r t n

Ignoring indexes for full-table scan - The MySQL server may, in some cases, calculate that a full table scan is cheaper to execute, even though it may fill the conditions for using indexes:

no a s a h ) one eindex • Multiple index operations - If a statement is designed ton utilize ฺ for a WHERE clause v ฺ d i and one index for an ORDER BY clause, the MySQL server will pick one of the multiple m Gu o operations to use and index for: c ฺ t 1 n 2 e s mysql> SELECT Name FROM City WHERE Id BETWEEN pt St10udAND 50 ORDER BY Name; s @ clause s ORDER BY clause exist, the MySQL server will use In the case where both a n WHERE iand h a t r the index for the ฺWHERE If no WHERE clause exists, the MySQL server may use the t clause. seThe n u index for the o ORDER BY clause. exception to this is if there is a composite index where the o o t first part can be used for the WHERE clause and the second part can be used for the ORDER BY m ( nse n clause: a r ce T i l t e SELECT Language, Percentage FROM CountryLanguage WHERE CountryCode = 'GBR' mysql> guy > ORDER BY Language; mysql> SELECT Name FROM City WHERE Id > 10;

N

•

Functions and Expressions - If a statement is designed with a function or expression in its WHERE clause, any indexed columns imbedded in the function or expression will not be utilized:

mysql> SELECT Name FROM Country WHERE LEFT(Code,1)='A';

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-9

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Quiz 2-A Objective

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Using the following table, evaluate each query and determine if the query optimizer will be able to use the indexes assigned:

Thi

CREATE TABLE t1 ( a INT, b INT, c INT, d INT, INDEX (a, b, c), INDEX (d) );

1.

SELECT d FROM t1 WHERE d = 10

Yes

No

2.

SELECT a FROM t1 WHERE a = 10

Yes

No

3.

SELECT c FROM t1 WHERE c = 10

4.

SELECT d FROM t1 WHERE d = 10 ORDER BY a

ns a r t n- Yes Yes

o an

s a h ) eฺ 6. SELECT a FROM t1 WHERE a = 10 AND b = 5ฺvn d i m Gu o c ฺ 7. SELECT a FROM t1 WHERE a = 10 AND 1 c = e17nt 2 s pt Stdud= 5 8. SELECT a FROM t1 WHERE as = 10 AND @ this n a r ฺt WHERE 9. SELECT a FROMnt1 sae= 1 AND b = 1 AND c = 5 u o o o tWHERE m ( e 10. SELECT a FROM t1 a = 1 AND b = 1 AND c = 5 AND d = 1 n ens a r c e11.t TSELECT lai FROM t1 WHERE a = 10 GROUP BY b 5.

N

guy

No

Yes

No

Yes

No

Yes

No

Yes

No

Yes

No

Yes

No

Yes

No

12. SELECT MAX(c) FROM t1 WHERE a = 10 GROUP BY b

Yes

No

13. SELECT a FROM t1 WHERE a = 10 AND d = 5 ORDER BY b

Yes

No

14. SELECT a FROM t1 WHERE a BETWEEN 1 AND 10 ORDER BY b

Yes

No

15. SELECT a FROM t1 WHERE a = 10 AND b < 3 ORDER BY c

Yes

No

SELECT a FROM t1 WHERE a = 10 ORDER BY b

e

bl a r fe No

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-10

MySQL Developer Techniques

2.5 47

Retrieving Index Information In MySQL, there are a multiple ways to retrieve information regarding the indexes that are assigned to tables within the databases. The following are the four most common approaches: •

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 2 - Improving Performance with Indexes

SHOW COLUMNS FROM - This command provides information about the columns in a table. The DESCRIBE command is a shortcut for the SHOW COLUMNS FROM command. These commands provide a quick view of the columns in the table referenced along with a small number of additional fields that provide data types of the columns, if the column can accept NULL values, if the column is indexed, the default value for the column and any additional information that are considered important concerning the column. The LIKE clause can be used with this SHOW command to limit the output to only those columns desired (ex. LIKE 'P%' to obtain on those columns that start with the letter 'P')

mysql> SHOW COLUMNS FROM City; +-------------+----------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+----------+------+-----+---------+----------------+ | ID | int(11) | NO | PRI | NULL | auto_increment | | Name | char(35) | NO | | | | | CountryCode | char(3) | NO | | | | | District | char(20) | NO | | | | | Population | int(11) | NO | | 0 | | +-------------+----------+------+-----+---------+----------------+ 5 rows in set (#.## sec)

e

bl a r fe

s

an r t n

T

no a s a h n) ideฺ v ฺ Key field valuesm u o G c ฺ t The Key field that is displayed in numerous of the commands1 n table and/or column information can have one 2 used to ddisplay e s of four values associated with it: t sp onlySastua secondary column in a multiple-column, non-unique • Empty - the column either is not indexed or is indexed s @ i n h index. a t r t e ฺ or is uonesof the columns in a multiple-column PRIMARY KEY. • PRI - the column is a PRIMARY onKEY o o • UNI - the column is the first columneofta unique-valued index that cannot contain NULL values. m ( s n multiple noccurrences athat • MUL - this means of a given value are allowed within the column. The column is the first column r e c T i l t of a non-unique index or a unique-valued index that can contain NULL values e y u g • SHOW CREATE TABLE - This command displays the SQL syntax that was used to create the N 48 i h table referenced. This can be useful in being able to see the indexing that is set up along with additional information such as the data types being used and any comments associated with the table.

mysql> SHOW CREATE TABLE City\G *************************** 1. row *************************** Table: City Create Table: CREATE TABLE `city` ( `ID` int(11) NOT NULL AUTO_INCREMENT, ... `Population` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`ID`), ) ENGINE=MyISAM AUTO_INCREMENT=4080 DEFAULT CHARSET=latin1 COMMENT='This stores the cities associated with the world database' 1 row in set (#.## sec)

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-11

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

49

Thi

•

Chapter 2 - Improving Performance with Indexes

SHOW INDEX FROM - This command returns detailed table index information to include the column indexes, the cardinality (estimate of the number of unique values in the index) and the type of index used.

mysql> SHOW INDEX FROM City\G *************************** 1. row *************************** Table: City Non_unique: 0 Key_name: PRIMARY Seq_in_index: 1 Column_name: ID Collation: A Cardinality: 4079 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment:

• 50

e

SELECT ... FROM INFORMATION_SCHEMA.STATISTICS - The MySQL data dictionary implementation (INFORMATION_SCHEMA) is the place that stores information about all the other databases that the MySQL server maintains. The STATISTICS table provides information about table indexes. The output from this table is equivalent to the SHOW INDEX FROM command. The following SQL statement displays a few of the columns associated with the STATISTICS table to show the similarity with the SHOW INDEX FROM command.

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s mysql> SELECT INDEX_NAME, COLUMN_NAME, tSEQ_IN_INDEX, p Stu -> CARDINALITY, INDEX_TYPE s @ -> FROM INFORMATION_SCHEMA.STATISTICS is n h a t -> WHERE TABLE_NAME='City' AND TABLE_SCHEMA='world'; r ฺt use n +-------------+-------------+--------------+-------------+------------+ o o | INDEX_NAME | COLUMN_NAME t|oSEQ_IN_INDEX | CARDINALITY | INDEX_TYPE | m ( e +-------------+-------------+--------------+-------------+------------+ s | | PRIMARY ran| ID en 1 | 4079 | BTREE | c T i +-------------+-------------+--------------+-------------+------------+ l etin set (#.## sec) 1 y row u g

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-12

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Lab 2-B

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this lab you will use some of the SQL commands presented to retrieve information concerning the world.CountryLanguage table.

Thi

Step 1: Verify that you are in the world database In a the mysql client, type the following commands: mysql> USE world; mysql> SHOW TABLES; +-----------------------+ | Tables_in_world_clean | +-----------------------+ | city | | country | | countrylanguage | +-----------------------+ 3 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v mysql> DROP DATABASE world; ฺ m Gu o c ฺ t mysql> CREATE DATABASE world; 1 n 2 e s pt Stud s mysql> USE world; @ this n a r mysql> SOURCE C:/world.sql ฺt use n o o tofile may different than the one displayed, please change it accordingly. The location of(m the world.sql e n indexes nsof the CountryLanguage table using MySQL SHOW commands e StepT2:ra View theic l t e y In the mysql client, the following command to view the columns and associated information using gu the SHOW COLUMNStypeFROM command:

You may have more tables than shown here in your output. This can be corrected by removing the world database and recreating it with the following commands:

N

mysql> SHOW COLUMNS FROM CountryLanguage; +-------------+---------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------------+---------------+------+-----+---------+-------+ | CountryCode | char(3) | NO | PRI | | | | Language | char(30) | NO | PRI | | | | IsOfficial | enum('T','F') | NO | | F | | | Percentage | float(4,1) | NO | | 0.0 | | +-------------+---------------+------+-----+---------+-------+ 4 rows in set (#.## sec)

You may wish to shorten the characters typed by using the DESC command (which is the shorter version of the DESCRIBE command which is equivalent): DESC CountryLanguage; ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-13

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Utilize the SHOW CREATE TABLE command to see even more detailed information concerning the CountryLanguage table to include the indexes associated with the table:

Thi

mysql> SHOW CREATE TABLE CountryLanguage\G *************************** 1. row *************************** Table: CountryLanguage Create Table: CREATE TABLE `countrylanguage` ( `CountryCode` char(3) NOT NULL DEFAULT '', `Language` char(30) NOT NULL DEFAULT '', `IsOfficial` enum('T','F') NOT NULL DEFAULT 'F', `Percentage` float(4,1) NOT NULL DEFAULT '0.0', PRIMARY KEY (`CountryCode`,`Language`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 1 row in set (0.00 sec)

This explains the reason that both the CountryCode and the Language columns were both identied as primary keys in the SHOW COLUMNS command. From this output, it is easy to see that both of these columns are used to make up the primary key. Utilize the last of the SHOW commands presented by typing the following command in the mysql client:

e

bl a r fe

s

mysql> SHOW INDEX FROM CountryLanguage\G *************************** 1. row *************************** Table: CountryLanguage Non_unique: 0 Key_name: PRIMARY Seq_in_index: 1 Column_name: CountryCode Collation: A Cardinality: NULL Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: *************************** 2. row *************************** Table: CountryLanguage Non_unique: 0 Key_name: PRIMARY Seq_in_index: 2 Column_name: Language Collation: A Cardinality: 984 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: 2 rows in set (#.## sec)

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

The reason the first column does not show a cardinality is due to the fact that it is the first part of a multicolumn primary key. In this case, the cardinality will be associated with the last column of the multicolumn primary key.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-14

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Step 3: View the indexes of the CountryLanguage table using INFORMATION_SCHEMA database

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In the mysql client, type the following command to view the indexes associated with the world.CountryLanguage table using the INFORMATION_SCHEMA database:

Thi

mysql> SELECT INDEX_NAME, COLUMN_NAME, SEQ_IN_INDEX, -> CARDINALITY, INDEX_TYPE -> FROM INFORMATION_SCHEMA.STATISTICS -> WHERE TABLE_NAME='CountryLanguage' AND TABLE_SCHEMA='world'; +------------+-------------+--------------+-------------+------------+ | INDEX_NAME | COLUMN_NAME | SEQ_IN_INDEX | CARDINALITY | INDEX_TYPE | +------------+-------------+--------------+-------------+------------+ | PRIMARY | CountryCode | 1 | NULL | BTREE | | PRIMARY | Language | 2 | 984 | BTREE | +------------+-------------+--------------+-------------+------------+ 2 rows in set (#.## sec)

e

bl a r fe

From the SEQ_IN_INDEX column, it is easy to see which column is first in the multi-column primary key index and which one is second. Again, this command is equivalent to the SHOW INDEX command but for many, the INFORMATION_SCHEMA tables are easier to work with.

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-15

MySQL Developer Techniques

2.6

Optimize Indexes Due to the potential for indexes to be a source of great return on investment when it comes to MySQL performance, the following “tricks” of index optimization are presented:

51

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

•

Thi

Chapter 2 - Improving Performance with Indexes

Prefix of Column Indexes- An index on a whole column is not always necessary. Instead, it may be best to create an index on just a prefix of a column. Prefix indexes take less space and the operations are faster. Prefix index for data selective by first few characters to help the optimizer determine statistically if it is useful for a given query.

Character types only Prefixing of column indexes works only with character data types. In addition, prefixes usually produce a noticeable time saving when applied to longer character data types. •

Composite Indexes - Composite indexes can be used for searches on the first column(s) in the index. In addition, if a query is using MIN, MAX or ORDER BY on secondary (or subsequent) columns it can also see improvements in performance due to the composite index.

e

bl • Primary Key – Minimizing the size of the primary keys that are used as references in other tables ra fe can gain incremental performance benefits. Additionally, using an AUTO_INCREMENT column s n can be more optimal for the primary key column. tra n • Covering index - A covering index, which may be a composite index, includes of the columns noofall this, referenced in the SELECT, JOIN, and WHERE clauses of a query. Because the index a s a contains the data being looked for and MySQL server doesn't have to look up the actual data in h ) the table, reducing logical and/or physical I/O, thus boosting performance. While non-covering ฺ ฺvn indexes composite indexes can hinder performance, covering composite idecan be very useful, and in m u o many cases, really boost the performance of a query data reads. G ฺc by saving t 1 n 2 e pts Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-16

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Further Practice 2-C

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this further practice, you will utilize the fundamentals of improving indexes that were presented in this chapter.

Thi

1. 2.

If necessary, install a fresh copy of the world database to remove any changes that may have been made to the data or the data structures. Create three new indexes in the world.Country table: a.

A unique index named uName that indexes the Name column.

b.

A composite index named Cont_Pop that indexes the Continent and Population columns together. A unique index named uCapital that indexes the Capital column.

c. 3. 4.

Create a new index in the world.City table against the CountryCode column. Determine which indexes (if any) the MySQL query optimizer would consider using in the following SQL statement:

e

bl a r fe

s

an r t n

mysql> SELECT DISTINCT Country.Region FROM Country INNER JOIN CountryLanguage ON > CountryLanguage.CountryCode = Country.Code WHERE Language = 'Spanish';

N

no a Based on the query execution plan, is the CountryLanguage table thes first to be evaluated (to a h look up the language 'Spanish')? Why or why not? n) ideฺ v ฺ ________________________________________________________________________________ m Gu o c ฺ t ________________________________________________________________________________ 1 n 2 e s ud optimizer would consider using in the 5. Determine which indexes (if any)sthe ptMySQLSquery t following SQL statement: @ this n a r ฺt FROM seCountry WHERE Population > 200000000; n mysql> SELECT Name, Continent u o o to m ( e s Basedn on the query plan, are there any indexes that could be utilized in the execution of norexecution aquery? r e this Why why not? c T li t e y ________________________________________________________________________________ gu ________________________________________________________________________________ Modifying the above SQL statement, determine which indexes (if any) the MySQL query optimizer would consider using now: mysql> SELECT DISTINCT Continent FROM Country WHERE Population > 200000000;

Based on the output, would this modified SQL statement change the query execution plan? Why or why not? ________________________________________________________________________________ ________________________________________________________________________________

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-17

MySQL Developer Techniques 6.

Chapter 2 - Improving Performance with Indexes

Determine which indexes (if any) the MySQL query optimizer would consider using in the following SQL statement:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

mysql> SELECT CountryCode, MAX(Population) FROM City GROUP BY CountryCode;

Thi

Would the CountryCode index, which was created in step three, be considered in the execution of this query? Why or why not? ________________________________________________________________________________ ________________________________________________________________________________ Modifying the above SQL statement, determine which indexes (if any) the MySQL query optimizer would consider using now: mysql> SELECT CountryCode, MAX(Population) FROM City WHERE CountryCode LIKE 'C%' > GROUP BY CountryCode;

e

bl a r fe

s

Based on the output, would this modified SQL statement change the query execution plan? Why or why not?

an r t n

no a ________________________________________________________________________________ s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a lice t Tr ________________________________________________________________________________

e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-18

MySQL Developer Techniques

2.7

Summary In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

52

Thi

Chapter 2 - Improving Performance with Indexes

•

Describe how MySQL handles query executions

•

Explain why MySQL uses indexes

•

List when MySQL uses and index and when it does not use an index

•

Describe ways to retrieve index information

•

Explain processes to optimize indexes

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

2-19

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 3

e

s

bl a r fe

an r t n

no SEARCHING DATE a s a h n) ideฺ v AND TEXT oFIELDS ฺ m u

uy g N

ฺc nt G 1 2 e pts Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

3 3.1

SEARCHING DATE AND TEXT FIELDS Overview This chapter provides the knowledge and skills necessary to improve the writing and use of queries that search date and text fields. At the completion of this chapter, you will be able to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

54

Thi

Chapter 3: Searching Date and Text Fields

•

Search for a string in multiple columns

•

Compare case sensitivity for text and binary searches

•

Utilize INET_ATON and INET_NTOA functions

•

Convert string dates to SQL dates

•

Convert dates to integer values

•

Convert a date to a string

e

s

Search for a non-specific date

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e •

bl a r fe

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-1

MySQL Developer Techniques

3.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

55

Chapter 3: Searching Date and Text Fields

Strings in Multiple Columns There are times when developers (either by inheriting existing code or attempting to simplify their own code) will create SQL statements that look harmless but turn out to be anything but harmless. Searching for strings in multiple columns can lead to many expensive SQL statements that cost the end user in time and performance. CONCAT in a WHERE clause Once of the ways that developers have avoided using multiple search means to come up with the result set they are looking for is by using the CONCAT function in a WHERE clause. For example, to find the rows that contain any users with a first name that begins with the letter 'M' and contains the last name 'Keitel', it might make sense to use a CONCAT function:

mysql> SELECT actor_id FROM actor -> WHERE CONCAT(LEFT(first_name,1),last_name) = 'MKeitel'; +----------+ | actor_id | +----------+ | 74 | | 198 | +----------+ 2 rows in set (#.## sec)

e

bl a r fe

s

an r t n

T

no a s a h This would produce the result intended, but it might be a good practice to determine how the database ) iteisฺpossible nfunction, 56 v server actually performed the requested query. With the EXPLAIN to remove the ฺ d iserver. m u guess work and see exactly how the queries are going to be executed by the o ฺc nt G 1 2 ts WHEREtude mysql> EXPLAIN SELECT actor_id FROM actor p s -> CONCAT(LEFT(first_name, 1), last_name) @ this S = 'MKeitel'\G *************************** a 1.nrow *************************** tr se id: 1 ฺ n u select_type: SIMPLEo o o t table: actor m ( nse type: n ALL a r possible_keys: NULL ce T i l t key: NULL NULL uye key_len: g ref: NULL N rows: 200 hi Extra: Using where 1 row in set (#.## sec)

Using the CONCAT function results in a problem, are there any keys that could be used that are not being used? This query would result in a full table scan, which for all practical purposes is not the most effective means to return results.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

57

Chapter 3: Searching Date and Text Fields

Now that this query did not work as well as expected, it is time to try a more traditional means to search multiple columns: mysql> EXPLAIN SELECT actor_id FROM actor > WHERE LEFT(first_name, 1) = 'M' AND last_name = 'Keitel'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: actor type: ref possible_keys: idx_actor_last_name key: idx_actor_last_name key_len: 137 ref: const rows: 3 Extra: Using where 1 row in set (#.## sec)

e

bl a r The results of this query would take advantage of an index containing the idx_actor_last_name sfe key and improve the response time of a large data set immensely by avoiding a full table scan. a So,neven -trapproach. though it may look cleaner, using the CONCAT function in the WHERE clause is not then best o However, is the LEFT function in this WHERE clause still limiting the query from n reaching its full a potential. The last query to review with the EXPLAIN function may be the best possible solution to take s a 58 h full advantage of all possible indexes: n) ideฺ v ฺ u mysql> EXPLAIN SELECT actor_id FROM actor o=m'Keitel'\G G c ฺ t > WHERE first_name LIKE 'M%' AND last_name 21 den *************************** 1. row *************************** s t id: 1 sp s Stu select_type: SIMPLE @ i n h table: actor a t r t e type: ref onฺ to us possible_keys: idx_actor_last_name,idx_actor_first_name o key: idx_actor_last_name (m nse n key_len: 137 a rref: const ce T i l t rows: 3 uye Extra: Using where g N 1 row in set (#.## sec) i h

T

The potential results may not have changed much for this particular query, but removing the LEFT function did provide the MySQL server with another possible key to utilize in providing the most efficient results.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-3

MySQL Developer Techniques

3.3

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

59

Chapter 3: Searching Date and Text Fields

Case Sensitivity Any text search in MySQL is based on the character set and collation that is being used. You can review which collations are being used in the server by issuing the following SQL statement:

mysql> SHOW VARIABLES LIKE 'coll%'; +----------------------+-------------------+ | Variable_name | Value | +----------------------+-------------------+ | collation_connection | latin1_swedish_ci | | collation_database | latin1_swedish_ci | | collation_server | latin1_swedish_ci | +----------------------+-------------------+ 3 rows in set (#.## sec)

Types of searches Searches can be made in a case sensitive, case insensitive or binary fashion. The corresponding collations will end in _cs, _ci and _bin. The following SQL statement demonstrates how a case insensitivity collation works:

e

bl a r fe

s

mysql> SELECT Code, Name FROM Country WHERE Code = 'Fin'; +------+---------+ | Code | Name | +------+---------+ | FIN | Finland | +------+---------+ 1 row in set (#.## sec)

an r t n

T

no a s a h n) ideฺ v ฺ om t Gu c ฺ 1 n 2 e s d t To alter the collation setting for the currentp 60 ttheu following SQL can be used: s connection, S @ s i n = latin1_general_cs; h mysql> SET COLLATION_CONNECTION a t r t e onฺ to us o With the current (mconnections e collation set to case-sensitive, the following SQL should produce an empty s n set: n a r ce T i l t ye SELECT Code, Name FROM Country WHERE Code = 'Fin'; umysql> +------+---------+ g hi N | Code | Name | +------+---------+ | FIN | Finland | +------+---------+ 1 row in set (#.## sec)

The fact that the SQL statement actually worked and did not produce an empty set could be an issue if it was not for the fact that each column's defined collation can override the collation of the connection (or table, database, server, etc.).

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-4

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this case, the Country.Code column has a collation that is different from the current connections collation:

Thi

mysql> SHOW FULL COLUMNS FROM Country WHERE Field='Code'\G *************************** 1. row *************************** Field: Code Type: char(3) Collation: latin1_swedish_ci Null: NO Key: PRI Default: Extra: Privileges: select,insert,update Comment: 1 row in set (#.## sec)

61

e

bl a r fe

CHARSET defaults When a DEFAULT CHARSET clause is part of the table creation syntax and there is no defined collation, a default collation will be implied based on the defined character set. The default collation for the latin1 character set (and all character sets) can be seen by issuing the following SQL statement:

s

an r t n

o n a mysql> SHOW COLLATION; s +----------------------+----------+-----+---------+----------+---------+ a h | Collation | Charset | Id | Default | Compiled n) i|dSortlen eฺ | v +----------------------+----------+-----+---------+----------+---------+ ฺ u| | big5_chinese_ci | big5 | 1 | Yes om | Yes 1 | G c ฺ t ... ... ... 1 n | latin1_swedish_ci | latin1 | 8 | 1 | s|2Yes tud|eYes t p ... ... ... s S +----------------------+----------+-----+---------+----------+---------+ @ s i n h 126 rows in set (#.## sec) ra t t e ฺ s n o to u odisplay, As seen in (this the default collation connected to the latin1 character set is m eThus, s n latin1_swedish_ci. the collation is case insensitive for the table no matter what the n a r e c T i connection collation may be. l et y u g

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-5

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Alter a column's collation The collation for a column can be specifically set in the table definition by identifying the collation to be used either in the table creation or afterwards by altering the table:

62

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

mysql> ALTER TABLE Country MODIFY Code CHAR(3) COLLATE latin1_general_cs;

Thi

This SQL has overwritten the default collation and defined a case sensitive collation to the Country table. Issuing the same SQL presented earlier will now produce the intended result of an empty set: mysql> SELECT Code, Name FROM Country WHERE Code = 'Fin'; Empty set (#.## sec)

To actually return a result, the SQL must be specifically set to include case sensitivity when searching against the table. The following SQL takes into account the case sensitive collation that the Country table is using: mysql> SELECT Code, Name FROM Country WHERE Code = 'FIN'; +------+---------+ | Code | Name | +------+---------+ | FIN | Finland | +------+---------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-6

MySQL Developer Techniques

3.3.1

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

63

Thi

Chapter 3: Searching Date and Text Fields

Binary searches The difference between binary comparisons versus collation based (text searches) is that there is no need to consider sort order or multiple characters evaluating to the same sort-value. Thus binary comparisons will produce better performance; however, all the specific features associated with language and character sets will not be available. Binary searches are accomplished by forcing one side of the comparison operation to become binary thus forcing the whole comparison into a binary context. To demonstrate how this would work, the following SQL returns the Country table back to a case insensitive collation:

mysql> ALTER TABLE Country MODIFY Code CHAR(3) COLLATE latin1_general_ci;

With the Country table being case insensitive again, the following SQL will produce the desired output without taking into account that the actual search does not have the same case as the stored data: mysql> SELECT Code, Name FROM Country WHERE Code = 'Fin'; +------+---------+ | Code | Name | +------+---------+ | FIN | Finland | +------+---------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ By adding the BINARY clause to either side of the comparison uthe MySQL server is forced to om operation, G c make the SELECT statement a binary comparison operation: ฺ t 21 den s t tu = BINARY 'Fin'; mysql> SELECT Code, Name FROM Country Code spWHERE S Empty set (#.## sec) @ s i n h a t r t e mysql> SELECT Code, Name us WHERE BINARY Code = 'Fin'; onฺFROMtoCountry o Empty set (#.## sec) (m nse n a r ce T i l t e guy

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-7

MySQL Developer Techniques

Possible poor performance There is a possible down side to using the BINARY clause with an SQL statement that could utilize an index that is associated with the content of the comparison operation. The following SQL describes how the SQL server would execute the binary comparison operation previously shown:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

64

Thi

Chapter 3: Searching Date and Text Fields

mysql> EXPLAIN SELECT Code, Name FROM Country WHERE BINARY Code = 'Fin'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: Country type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 239 Extra: Using where 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

Even though the Code column in the Country table is the primary key, the binary comparison operation will perform a full table scan because the index was not created for binary comparisons. This can be corrected by utilizing a binary collation.

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-8

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Inline Lab 3-A

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this lab you will search for values in multiple columns from the world.CountryLanguage table.

Thi

Step 1: Verify that you are in the world database In a the mysql client, type the following commands: mysql> USE world; mysql> SHOW TABLES; +-----------------------+ | Tables_in_world_clean | +-----------------------+ | city | | country | | countrylanguage | +-----------------------+ 3 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v mysql> DROP DATABASE world; ฺ m Gu o c ฺ t mysql> CREATE DATABASE world; 1 n 2 e s pt Stud s mysql> USE world; @ this n a r mysql> SOURCE C:\world.sql ฺt use n o o tofile may different than the one displayed, please change it accordingly. The location of(m the world.sql e n indexes nsof the CountryLanguage table using MySQL SHOW commands e StepT2:ra View theic l t e y In the mysql client, the following command to view the columns and associated information using gu the SHOW COLUMNStypeFROM command:

You may have more tables than shown here in your output. This can be corrected by removing the world database and recreating it with the following commands:

N

mysql> SHOW COLUMNS FROM CountryLanguage; +-------------+---------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------------+---------------+------+-----+---------+-------+ | CountryCode | char(3) | NO | PRI | | | | Language | char(30) | NO | PRI | | | | IsOfficial | enum('T','F') | NO | | F | | | Percentage | float(4,1) | NO | | 0.0 | | +-------------+---------------+------+-----+---------+-------+ 4 rows in set (#.## sec)

You may wish to shorten the characters typed by using the DESC command (which is the shorter version of the DESCRIBE command which is equivalent): DESC CountryLanguage; ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-9

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Step 3: Search the columns of the CountryLanguage table using functions

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In the mysql client, type the following command to view how the MySQL server will execute the SELECT statement that is utilizing functions in the WHERE clause:

Thi

mysql> EXPLAIN SELECT CountryCode FROM CountryLanguage -> WHERE LEFT(CountryCode,1)='D' AND Language = 'German'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: CountryLanguage type: index possible_keys: NULL key: PRIMARY key_len: 33 ref: NULL rows: 984 Extra: Using where; Using index 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

Even though the EXPLAIN output shows that there are indexed columns that are part of the WHERE clause, none of the indexes can be used due to a function being used. However, reviewing how the MySQL server will execute the following SELECT statement reveals something interesting:

o

N

an s ha ฺ mysql> EXPLAIN SELECT CountryCode FROM CountryLanguage n) ฺv uide -> WHERE CountryCode LIKE 'D%' AND LEFT(Language,1)='G'\G m o *************************** 1. row *************************** ฺc nt G id: 1 1 2 e select_type: SIMPLE pts Stud table: CountryLanguage s @ this type: range n a possible_keys: PRIMARY r t se key: PRIMARY nฺ u o o key_len: 3 to m ( ref: NULL e n 23 ens rows: a r T Using lic where; Using index tinExtra: e 1 y row set (#.## sec) gu

Here, the left portion (CountryCode) of the composite index (CountryCode,Language) is not using a function and thus the index can be used for this SELECT statement. Of course, not using the function in the second portion of the index would have a positive affect on a larger table.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-10

MySQL Developer Techniques

3.4 65

Chapter 3: Searching Date and Text Fields

INET_ATON and INET_NTOA Storing network addresses can be made more efficient by using the INET_ATON and INET_NTOA functions.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

INET_ATON (InterNET Address TO Network address)

Thi

Given the dotted-quad representation of a network address as a string, this function returns an integer that represents the numeric value of the address. A 4-byte address (213.136.52.29 as an example) would be represented in the following binary depiction: 11010101.10001000.00110100.00011101 (4-byte representation) The following is an example of how a 4-byte IP address would be converted using INET_ATON: mysql> SELECT INET_ATON('213.136.52.29'); +----------------------------+ | INET_ATON('213.136.52.29') | +----------------------------+ | 3582473245 | +----------------------------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

o n a The generated number is always in network byte order. For the example just s shown, the number is a calculated as 213×256 + 136×256 + 52×256 + 29. h n) ideฺ v What is this number? ฺ u oism c The number that is returned (in this case 3582473245), a 10 t G ฺ n 21 that dcame e number base representation of the binary numbers s t u p SQL tstatement from the IP address. For example, the following srepresentation S converts the number returned to an binary of the 10 @ s i h a t based number: r ฺt use n o o to mysql> SELECT CONV(3582473245,10,2); m ( e +----------------------------------+ an icens r | CONV(3582473245,10,2) | T l t +----------------------------------+ e y11010101100010000011010000011101 | gu|+----------------------------------+ 3

N

2

1 row in set (#.## sec)

Here the number provided by the INET_ATON function is converted from the 10 base to binary representation which connects with the IP address. In this case the returned result is 11010101100010000011010000011101 which is equivalent to 11010101.10001000. 00110100.00011101 which is equivalent to 213.136.52.29.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-11

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Short form IP addresses

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

66

Thi

In addition to the 4-byte addresses, the INET_ATON function understands short form IP addresses: mysql> SELECT INET_ATON('127.0.0.1'), INET_ATON('127.1'); +------------------------+--------------------+ | INET_ATON('127.0.0.1') | INET_ATON('127.1') | +------------------------+--------------------+ | 2130706433 | 2130706433 | +------------------------+--------------------+ 1 row in set (#.## sec)

Storing INET_ATON values When storing values generated by INET_ATON(), it is recommended that you use an INT UNSIGNED column. If you use a (signed) INT column, values corresponding to IP addresses for which the first octet is greater than 127 cannot be stored correctly.

s

an r t n

INET_NTOA (InterNET Network TO Address)

o

Given a numeric network address, returns the dotted-quad representation of the address as a string.

an s a mysql> SELECT INET_NTOA(3582473245); h ) +-----------------------+ n ideฺ v ฺ | INET_NTOA(3582473245) | om t Gu +-----------------------+ c ฺ | 213.136.52.29 | 21 den s +-----------------------+ t 1 row in set (#.## sec) sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e guy

e

bl a r fe

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-12

MySQL Developer Techniques

3.5

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

67

Chapter 3: Searching Date and Text Fields

Searching Dates Similar to strings in multiple columns, dates can become an issue for developers who wish to improve the look and effectiveness of their code. The issue is which searches provides the results required but also using the most effective means to return those results. Month and year One of the most common searches against a date column is obtaining all the rows for a specific month in a specific year. Now like any good programming language, SQL provides multiple ways to obtain these results. The first way that some developers have attempted to search for dates is to use a similar LIKE function that is used with strings:

mysql> EXPLAIN SELECT inventory_id FROM rental WHERE return_date LIKE '2006-02%'; Warning (Code 1292): Incorrect datetime value: '2006-02%' for column 'return_date' at row 1

e

T

bl a r Not exactly the output desired. This of course is an incorrect statement in relation to dates. Even though fe s n this approach may seem logical, it is not proper SQL. ra t n YEAR and MONTH functions no 68 a The next possible solution for solving the searching for dates dilemma is to date functions in the s useseems a WHERE clause. Using the YEAR and MONTH functions in the WHEREh clause like a reasonable ) ฺ n solution, but what does EXPLAIN say: ฺv uide m o ฺc nt G mysql> EXPLAIN SELECT inventory_id FROM rental 1 2 -> WHERE YEAR(return_date)=2006 AND de u ptsMONTH(return_date)=2\G t *************************** 1. row s *************************** S @ s id: 1 i n h select_type: SIMPLE ฺtra use t table: rentalon type: ALL o to m ( e possible_keys: NULL ns key: an NULL r e c T NULL li tkey_len: e ref: NULL y rows: 15609 gu N Extra: Using where i h 1 row in set (#.## sec)

This query will result in a full table scan, no indexes will be evaluated due to the functions modifying the value of return_date. This would result in the correct output but a very expensive means to achieve it. Could a more creative approach work better.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-13

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Getting between the dates

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

69

The most likely solution to obtain the results expected while also utilizing an available indexes is by using the BETWEEN clause to evaluate a date between two dates. In this case, the dates are the beginning of the month and the end of the month: mysql> EXPLAIN SELECT inventory_id FROM rental -> WHERE return_date BETWEEN '2006-02-01' AND '2006-02-28'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: rental type: range possible_keys: return_date key: return_date key_len: 8 ref: NULL rows: 2614 Extra: Using where 1 row in set (#.## sec)

e

bl a r fe

T

ns a r t Success, indexes are used. In this case, the rows will be evaluated utilizing o annindex with a n Here are some return_date key; however, do we actually get the results expected. Maybe not. a s issues to consider: a 70 h ) ฺ into account those nwould inot • Leap years - The dreaded month of February. This query take e v ฺ d years where February 29 appears. om t Gu c ฺ 1 is a DATETIME n • Last day of the month - Since return_date column, a DATE implicitly 2 e s d t converted to DATETIME is midnight on that date. This would result in not seeing any rentals u p t s ands2006-02-28 S between 2006-02-28 00:00:00 23:59:59. @ i n h a t r Compensating for these potential sethe following query would result in the correct output but void nฺt issues, u the index usage: oo to m ( e s ninventory_id an SELECT mysql> EXPLAIN FROM rental WHERE return_date r e c T i l ->t BETWEEN '2006-02-01 00:00:00' AND AS DATETIME) - INTERVAL 1 SECOND \G ye> CAST('2006-03-01 00:00:00' u*************************** g 1. row *************************** id: 1 hi N th

select_type: SIMPLE table: rental type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 16352 Extra: Using where 1 row in set (#.## sec)

This query takes into account every possible second of the last day of February by utilizing the first day of March minus 1 second. This may be extreme, but there is no chance of missing data or incorrect data being displayed. ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-14

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

End of month report

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

71

The next logical step to consider with the rental query up to this point is running it every single month. This is a common practice and using some creative (and valid) SQL can ensure that the results are easily and accurately obtained. The following SQL uses three functions called DATE_FORMAT, CURDATE and LAST_DAY in the WHERE clause that provides a valid date for the query: mysql> -> -> -> -> -> -> -> -> ->

SELECT inventory_id FROM rental WHERE return_date BETWEEN CAST(DATE_FORMAT( CURDATE(), '%Y-%m-01 00:00:00' ) AS DATETIME) AND CAST(DATE_FORMAT( LAST_DAY(CURDATE()),'%Y-%m-%d 23:59:59' ) AS DATETIME);

e

bl a r fe

s

an r t n

This query would take the current day (no matter what day it is in the month) and determine what the first day of the month would be (using the DATE_FORMAT function) and the last day of the month would be (using the LAST_DAY function). These two values are then connected to their proper time for each day and then that data is evaluated to produce the correct and accurate output. This could easily be stored in a stored procedure or an external application to eliminate having to type this at the end of every month.

no a s a h n) ideฺ v ฺ com nt Gu ฺ Is CAST()1needed? s2however, diteis a good practice to be explicit when you work t The CAST() function in the above queries is not needed; u p t s S the right thing. The CAST() function will return results with data type conversions rather than hope that the server does @ s i n h as strings. tra se t ฺ n o to u o m n ( ense a r T lic t e uy g hi N

T

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-15

MySQL Developer Techniques

3.6

String Dates to SQL Dates Converting a string date to an actual date within MySQL can be accomplished with the STR_TO_DATE function. This function is the opposite of the DATE_FORMAT function. The STR_TO_DATE function takes a string date as the first argument followed by the format type (of the string date) as the second argument. The following demonstrates how the this function works:

72

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 3: Searching Date and Text Fields

mysql> SELECT STR_TO_DATE('07/14/2007','%m/%d/%Y'); +--------------------------------------+| | STR_TO_DATE('07/14/2007','%m/%d/%Y') | +--------------------------------------+ | 2007-07-14 | +--------------------------------------+ 1 row in set (#.## sec)

e

bl a r fe

The second argument in this function tells the STR_TO_DATE function that the characters up to the first forward slash ('07') identify the month, the characters between the first forward slash and the second forward slash ('14') identify the day and the 4 characters after the second forward slash ('2007') identify the 4-digit year. The MySQL server then displays the date in the ISO format for dates.

s

no a The following chart displays the specifier syntax and the description for the most s common date and time a h format types. n) ideฺ v ฺ Specifier Description om t Gu %a Abbreviated weekday name (Sun..Sat) c ฺ 1 n %b Abbreviated month name (Jan..Dec) 2 e s d t %c p Stu Month, numeric (1..12) s %D @ s ... 31st) Day of the month with English suffix i(1st n h a t %d r Day of the month, e ฺt numeric s(01..31) n %e u Day of the month, numeric (1..31) o o (000000..999999) to %f m Microseconds ( e %H (00..23) ns an Hour r e c T li numeric (00..59) Minutes, t%i e y %j Day of year (001..366) u Format Specifiers

g

N Thi

an r t n

%k

Hour (0..23)

%l %M

Hour (1..12) Month name (January..December)

%m

Month, numeric (01..12)

%r %S

Time, 12-hour (hh:mm:ss followed by AM or PM) Seconds (00..59)

%T %U

Time, 24-hour (hh:mm:ss) Week (01..53), where Sunday is the first day of the week

%W

Weekday name (Sunday..Saturday)

%w %Y

Day of the week (0=Sunday..6=Saturday) Year, numeric, four digits

%y %%

Year, numeric (two digits) A literal “%” character

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-16

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

73

Chapter 3: Searching Date and Text Fields

Illegal entries In the event that the values entered in the first argument for the STR_TO_DATE function do not line up with the format expected from the second argument, the response will be a NULL value: mysql> SELECT STR_TO_DATE ('07/14/2007','%y/%m/%d'); +---------------------------------------+ | STR_TO_DATE ('07/14/2007','%y/%m/%d') | +---------------------------------------+ | NULL | +---------------------------------------+ 1 row in set, 1 warning (#.## sec) mysql> SHOW WARNINGS; +-------+------+-----------------------------------------------------------------+ | Level | Code | Message | +-------+------+-----------------------------------------------------------------+ | Error | 1411 | Incorrect datetime value: '07/14/2007' for function str_to_time | +-------+------+-----------------------------------------------------------------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

T

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t p Stu s No TIME like the present @ 74 is n h a t r The resulting data type from function is the DATETIME data type. This of course ฺt the STR_TO_DATE se can handle n u implies that the STR_TO_DATE function time string conversions also. The following o o o t example demonstrates how this would work: m n ( ense a r T STR_TO_DATE lic mysql>t SELECT ('1:23 PM','%l:%i %p'); e y +------------------------------------+ gu| STR_TO_DATE ('1:23 PM','%l:%i %p') | N hi +------------------------------------+ In the STR_TO_DATE function above, the second argument tells the function to expect a two digit year in the first two characters of the string. That would have been met with no problems; however, the characters after the first forward slash and before the second forward slash ('14') are to contain a value that would be appropriate for a numeric month identifier. In this case, the numeric identifier '14' is not a valid month entry and causes the function to error out. The third set of characters after the second forward slash are ignored due to the function already erroring out; however, this would of also caused problems due to no month containing 2007 days.

| 13:23:00 | +------------------------------------+ 1 row in set (#.## sec)

In this example, the time of '1:23 PM' is converted to the ISO format of 13:23:00 using the format structure of hour (%1) followed by a colon and minutes (%i). In addition, due to this time not identifying if it is one in the morning or one in the afternoon, the string contains the identification of PM which is specified by %p in the format string.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-17

MySQL Developer Techniques

3.7

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

75

Chapter 3: Searching Date and Text Fields

Dates to Integer Values Within standard SQL, there is a function called TO_DAYS that converts a date into an integer value. The resulting integer value is the number of days that have occurred since the year 0000. The following example shows the integer value associated with the acquisition of MySQL AB by Sun Microsystems:

mysql> SELECT TO_DAYS('2008-02-26'); +-----------------------+ | TO_DAYS('2008-02-26') | +-----------------------+ | 733463 | +-----------------------+ 1 row in set (#.## sec)

Calculating date differences in days 76

Using the TO_DAYS SQL function, an end user can calculate the number of days between two different dates. The following example shows the length of days from the first release of MySQL (version 1.0 released internal to the company) to the acquisition of MySQL AB by Sun Microsystems:

e

bl a r fe

s

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t tu a similar result: spthat swillSproduce With standard SQL there is another function @ 77 i n h a t r t e mysql> SELECT DATEDIFF('2008-02-26','1995-05-23'); onฺ to us o +-------------------------------------+ (m nse | DATEDIFF('2008-02-26','1995-05-23') | n +-------------------------------------+ a r e T | 4662 | lic t e +-------------------------------------+ y gu1 row in set (#.## sec) N hi mysql> SELECT TO_DAYS('2008-02-26') - TO_DAYS('1995-05-23'); +-----------------------------------------------+ | TO_DAYS('2008-02-26') - TO_DAYS('1995-05-23') | +-----------------------------------------------+ | 4662 | +-----------------------------------------------+ 1 row in set (#.## sec)

an r t n

T

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-18

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

78

Chapter 3: Searching Date and Text Fields

Calculating date differences with a date The FROM_DAYS function displays the year, month and days associated with the integer value entered. The function can be considered the matching function for the TO_DAYS function: mysql> SELECT FROM_DAYS(4662); +-----------------+ | FROM_DAYS(4662) | +-----------------+ | 0012-10-06 | +-----------------+ 1 row in set (#.## sec)

This value displays a date which is equivalent to 12 years, 10 months and 6 days. This length of time is an approximate length that shows how long in a more readable format (versus 4,662 days) of the date difference from when MySQL version 1.0 was released until Sun Microsystems acquired MySQL AB. It is considered approximate because the months between and leap years are not taken into account exactly, so plus or minus up to 2 days should be considered the range. For any date difference that is 365 days or less, the following output will be the result:

e

bl a r fe

s

an r t n

T

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t Proleptic Gregorian calendar e ฺ s n u o o from thetoJulian to the Gregorian calendar has had to discard at least ten days during the Every country that has switched m ( switch. There are no dates between seOctober 4 and October 15. This discontinuity is called the cutover. Any dates before n n a r e the cutover are TJulian, andlicany dates following the cutover are Gregorian. Dates during a cutover are non-existent. t e y applied to dates when it wasn't actually in use is called proleptic. Thus, if we assume there was never a Au calendar g cutover and Gregorian rules always rule, we have a proleptic Gregorian calendar. This is what is used by MySQL, as is hi Nrequired by standard SQL. For this reason, dates prior to the cutover stored as MySQL DATE or DATETIME values must mysql> SELECT FROM_DAYS(365); +----------------+ | FROM_DAYS(365) | +----------------+ | 0000-00-00 | +----------------+ 1 row in set (#.## sec)

be adjusted to compensate for the difference. It is important to realize that the cutover did not occur at the same time in all countries, and that the later it happened, the more days were lost. For example, in Great Britain, it took place in 1752, when Wednesday September 2 was followed by Thursday September 14. Russia remained on the Julian calendar until 1918, losing 13 days in the process, and what is popularly referred to as its “October Revolution” occurred in November according to the Gregorian calendar.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-19

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Inline Lab 3-B

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this lab you will use some of the tools you learned in these last few sections to answer questions related to countries and the date of their independence from data stored in the Country table.

Thi

Step 1: Review the data type for the column we are going to reference Using the world database, type the following command in the mysql client: mysql> SHOW COLUMNS FROM Country LIKE 'I%'; +-----------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+-------------+------+-----+---------+-------+ | IndepYear | smallint(6) | YES | | NULL | | +-----------+-------------+------+-----+---------+-------+ 1 row in set (0.00 sec)

e

N

bl a r This shows that the year of indepence (IndepYear) for a country is an integer data type versus a date fe ns data type. Any queries involving this column and dates will need to take this into account. a r t n Step 2: Convert the IndepYear Integer data type to a valid date type o an With the IndepYear value being an integer that contains only the 4 digit year, converting the value to a s a h date takes some creative SQL. There are multiple ways to accomplish this. The following command is ) n ideฺ one way to accomplish this task: v ฺ u om FROM G c ฺ t mysql> SELECT IndepYear, FROM_DAYS(IndepYear*365.25) Country LIMIT 5; 1 n 2 e +-----------+-----------------------------+ s pt | Stud | IndepYear | FROM_DAYS(IndepYear*365.25) s +-----------+-----------------------------+ @ this| | 1919 | 1919-01-16 ran ฺt use | | 1581 | 1581-01-12 n o | NULL | NULL o | to m | 1912 | 1912-01-16 | ( e s | 1962a|n1962-01-16 | n r e ... c T i l t | ye NULL | NULL | u | NULL | NULL | g +-----------+-----------------------------+ 239 rows in set (#.## sec)

With the year being calculated to take into account leap years (the .25 value), a date value is created that is equivalent to a date early in the year of the independence year stored in the Country table.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-20

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Step 3: Calculate the number of years since independence of a country

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In the mysql client, type the following command to calculate the current date minus the independence year of a country to calculate how long an individual country has been independent:

Thi

mysql> SELECT IndepYear, DATEDIFF(NOW(),FROM_DAYS(IndepYear*365.25)) -> FROM Country; +-----------+---------------------------------------------+ | IndepYear | DATEDIFF(NOW(),FROM_DAYS(IndepYear*365.25)) | +-----------+---------------------------------------------+ | 1919 | 32787 | | 1581 | 156242 | | NULL | NULL | | 1912 | 35344 | | 1962 | 17081 | | NULL | NULL | ... | NULL | NULL | | NULL | NULL | +-----------+---------------------------------------------+ 239 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v mysql> SELECT IndepYear, ฺ -> DATEDIFF(NOW(),FROM_DAYS(IndepYear*365.25))/365.25 om t GASu Years_Indep c ฺ -> FROM Country LIMIT 10; 21 den +-----------+-------------+ s t | IndepYear | Years_Indep | sp s Stu +-----------+-------------+ @ i h | 1919 | 89.7659 |ran t t e | 1581 | 427.7673 nฺ || us oNULL | NULL | o o | 1912 | (m 96.7666e| t s| | 1962a|n 46.7652 n r e | NULL | NULL | c T li ... et NULL | gu| y NULL |

The output is the number of days difference, which of course is not very useful to most end users. A format of years would be more useful. This is accomplished with the following command:

N

| NULL | NULL | +-----------+-------------+ 239 rows in set (#.## sec)

Not only has the column name been changed to be easier to read, but also the data is easier to read with the addition of the year divisor.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-21

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Step 4: Cleaning up the output For the most part, the output is readable and could be used for multiple purposes if their was a need. However, cleaning up the output to minimize any irrelevant data (like an output of years with decimal values) would be more professional. The following command cleans up the output and provides additional value to that output. mysql> SELECT Name, -> CAST( -> (DATEDIFF( -> NOW(), FROM_DAYS(IndepYear*365.25) ) / 365.25 -> ) AS UNSIGNED -> ) AS Years_Indep -> FROM Country WHERE IndepYear IS NOT NULL; +----------------------+-------------+ | Name | Years_Indep | +----------------------+-------------+ | Afghanistan | 90 | | Netherlands | 428 | | Albania | 97 | | Algeria | 47 | | Andorra | 731 | | Angola | 34 | ... | Estonia | 18 | | United States | 233 | | Zimbabwe | 29 | +--------------- ------+-------------+ 192 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t tu sp s S This output provides the name of the country, then uses the CAST function to turn the Years_Indep @ n places)thfori only countries that have a value in the IndepYear output into an integer (no decimal a r t ฺ NOT uNULL). se Your output may be slightly different than the output column (IndepYear n IS o othe NOW() function which will affect the output based on when this displayed here m dueo to the use tof ( e statement is anrun.icens r T l t e y u

g

N Thi

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-22

MySQL Developer Techniques

3.8

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

79

Thi

Chapter 3: Searching Date and Text Fields

Dates to Strings Extracting portions of a date field to display a string (or integer) value can be accomplished with numerous SQL functions. The following list demonstrates a portion of these functions:

Function DAY() DAYOFMONTH()

Description Returns the day of the month (1-31) Synonym for DAY() function

Usage DAY('2008-02-26') DAYOFMONTH('2008-02-26')

Result 26 26

DAYNAME() DAYOFWEEK()

Returns the name of the weekday Returns the weekday index (1-7)

DAYNAME('2008-02-26') DAYOFWEEK('2008-02-26')

Tuesday 3

DAYOFYEAR() MONTH()

Returns the day of the year (1-366) Returns the month (1-12)

DAYOFYEAR('2008-02-26') MONTH('2008-02-26')

57 2

MONTHNAME() QUARTER()

Returns the name of the month Returns the quarter of the year

MONTHNAME('2008-02-26') QUARTER('2008-02-26')

February 1

WEEK() WEEKDAY()

Returns the week of the year Returns the weekday index (0-6)

WEEK('2008-02-26',3) WEEKDAY('2008-02-26')

8 1

YEAR() YEARWEEK()

Returns the 4-digit year Returns the year and week

YEAR('2008-02-26') YEARWEEK('2008-02-26',3)

e

bl a r fe

ns a 2008 r t n- 200808

no a s a h How is the first week of the year calculated? n) ideฺ v ฺ When using the WEEK() function, there are eight different modes that can be u used to determine how the weeks are om G c ฺ t calculated. The modes are entered into the second argument of the function and range from 0 (which states that if the 1week forethenyear using 2 first week of the year contains a Sunday, consider if the zero range of 0-53) to 7 (which states s t week tuofdthe year using aarange that if the first week contains a Monday, considersitpthe first S of 1-53). The other modes @ s (1-6) provide more options to fine tune n the WEEK() function. The WEEKDAY() function does not take a mode i h a t r t argument and is equivalent to WEEK(date,3) which states that if the first week contains more than 3 days and begins e ฺ usingusa range of 1-53. n o with a Monday, consider it the first week o to m ( e an icens r T l t e y u Ng

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-23

MySQL Developer Techniques

3.8.1

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

80

Thi

Chapter 3: Searching Date and Text Fields

Time values to Strings As seen, MySQL has numerous date functions that will extract specific values from the dates entered. In addition to date functions, MySQL offers multiple time functions that can be valuable to administrators and developers alike. •

TIME() - This function will return the time portion of a date (and/or time) value. This function should only be used with valid time values to assure accurate responses.

mysql> SELECT TIME('1955-11-12 22:04:00'); +-----------------------------+ | TIME('1955-11-12 22:04:00') | +-----------------------------+ | 22:04:00 | +-----------------------------+ 1 row in set (#.## sec)

•

e

bl a r fe

s

HOUR() - This function will return the hours for a day from 0 to 23 and greater if the value given contains a greater hour value.

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ • MINUTE() - This function ithe minutes from a given time in the range of 0-59. n will return h a t r t e mysql> SELECT MINUTE('1955-11-12 onฺ to us22:04:00'); o +-------------------------------+ (m n se | MINUTE('1955-11-12 22:04:00') | n a r e +-------------------------------+ c T li | et 4 | y +-------------------------------+ u g1 row in set (#.## sec) mysql> SELECT HOUR('1955-11-12 22:04:00'); +-----------------------------+ | HOUR('1955-11-12 22:04:00') | +-----------------------------+ | 22 | +-----------------------------+ 1 row in set (#.## sec)

N

•

SECOND() - This function will return seconds from a given time also in the range of 0-59.

mysql> SELECT SECOND('1955-11-12 22:04:05'); +-------------------------------+ | SECOND('1955-11-12 22:04:00') | +-------------------------------+ | 5 | +-------------------------------+ 1 row in set (#.## sec)

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-24

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

•

Thi

Chapter 3: Searching Date and Text Fields

MICROSECOND() - This function will return the microseconds from a given time expression as a number in the range from 0 to 999999.

mysql> SELECT MICROSECOND('1955-11-12 22:04:00.00078405'); +---------------------------------------------+ | MICROSECOND('1955-11-12 22:04:00.00078405') | +---------------------------------------------+ | 78405 | +---------------------------------------------+ 1 row in set (#.## sec)

3.8.1.1 Storing Microsecond Precision Times 81

Microseconds can not be stored into a column of any of the MySQL temporal data types. If an attempt is made to store a time value with a microsecond into a temporal data type, the microsecond value is discarded. Storing as a DOUBLE

e

bl a r fe

s

an r t n

One of the ways that a time value can be stored is by converting the time to a numeric Unix timestamp. The Unix timestamp evaluates the current time (or the time provided) into an unsigned integer that equates to the number of seconds since '1970-01-01 00:00:00'. This Unix timestamp can be evaluated by issuing the UNIX_TIMESTAMP function:

N

no a s a h n) ideฺ mysql> SELECT UNIX_TIMESTAMP('2008-02-26 09:00:00');ฺv +---------------------------------------+ om t Gu | UNIX_TIMESTAMP('2008-02-26 09:00:00') | 1ฺc n 2 e +---------------------------------------+ s d t | 1204034400 sp |s Stu +---------------------------------------+ @ i n h 1 row in set (#.## sec) a t r t e s umicrosecond onฺvalue, Once the time is an o integer the value can be added to this value and then stored in a o t m ( e data type value such assDOUBLE. For example, the following table could be used to store the n that ea web n page was requested by a browser and the actual time that the server took to atime requested r c T i l t the request and begin a response back to the browser: eevaluate y u g mysql> CREATE TABLE web_request ( -> web_page VARCHAR(100), -> request_time DOUBLE(16,6), -> response_time DOUBLE(16,6) -> ); Query OK, 0 rows affected (#.## sec)

With this table in place, statistics can begin to be evaluated for each web page requested to the microsecond (up to 6 values) based on the DOUBLE data type being used (16 precision of significant digits being stored, with up to 6 of those digits being stored after the decimal point).

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-25

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

82

Chapter 3: Searching Date and Text Fields

Continuing with the example, the web_request table that was created will be filled with a record that will store the requested time that a web page on the mysql.com domain server received the request for access to one of its pages and when the server started the response back to the browser requesting the page: mysql> INSERT INTO web_request VALUES ( -> 'www.mysql.com/training/catalog.html', -> UNIX_TIMESTAMP('2008-02-26 09:00:00')+.001356, -> UNIX_TIMESTAMP('2008-02-26 09:00:00')+.978650 -> ); Query OK, 1 row affected (#.## sec)

The following query will show how the record just added was actually stored in the web_request table: mysql> SELECT web_page, request_time, response_time FROM web_request; +-------------------------------------+-------------------+-------------------+ | web_page | request_time | response_time | +-------------------------------------+-------------------+-------------------+ | www.mysql.com/training/catalog.html | 1204034400.001356 | 1204034400.978650 | +-------------------------------------+-------------------+-------------------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

no a s astored h With these values, along with additional requests over a set period of) time, in the web_request ฺ n e v table, statistics can be evaluated and monitored. The following query uses the one record stored to ฺ d iand 83 mis requested u determine a simple length of time scenario from when c ao page when the server begins G ฺ t to respond in milliseconds: 1 n s2 tude t p s s ASS diff FROM web_request; mysql> SELECT response_time - request_time @ i n +----------+ h a t r t e | diff | onฺ to us +----------+ o | 0.977294 | (m nse +----------+ n a r (#.## e 1 row inTset csec) i l t uye g hi N

T

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-26

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

One of many

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

The solution to storing microseconds presented is one of many ways to accomplish this task. The following are a list of a few other ways to accomplish this:

Thi

● BIGINT - Using the BIGINT data type, the time value would be stored as an integer (which implies no decimals). This would be accomplished by multiplying the Unix timestamp by 100,000 to make room for the microseconds to be stored in the last six values of the integer: mysql> SELECT UNIX_TIMESTAMP('2008-02-26 09:00:00') * 100000 + 001356; +---------------------------------------------------------+ | UNIX_TIMESTAMP('2008-02-26 09:00:00') * 100000 + 001356 | +---------------------------------------------------------+ | 120403440001356 | +---------------------------------------------------------+ 1 row in set (#.## sec)

mysql> SELECT '2008-02-26 09:00:00' AS sec, 978650 AS micro; +---------------------+--------+ | sec | micro | +---------------------+--------+ | 2008-02-26 09:00:00 | 978650 | +---------------------+--------+ 1 row in set (#.## sec)

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

e

bl a r fe

● Two columns - Storing the date/time value in one column and the microseconds in a second column can eliminate the need to use Unix timestamps:

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-27

MySQL Developer Techniques

3.9

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

84

Chapter 3: Searching Date and Text Fields

Nonspecific Date Searches There are times when a request is made to provide data that is not specific to an actual date that can be searched easily using standard SQL. Such a request could be "Give me all the times that employees have been absent from work the third Monday of each month for the last year." For the most part, this could simply be an exercise in pulling out a calendar, looking for each third Monday of the months for the last year, writing an SQL statement that searches for the specific dates that were obtained from this manual view and then providing the results to the requestor. However, what if there was someway to figure this out logically and let SQL handle the majority of this tedious work. The Logic The first day (day of the week) that a month begins on provides a lot of information that can be used to create the logic that can be used to obtain non-specific dates. The following chart shows how the first day of the month provides valuable information about the 1st and 3rd Mondays of the month: First Day of Month

# Weekday

1st Monday 3rd Monday

Sunday Monday Tuesday

1 2 3

2 1 7

Wednesday Thursday Friday

4 5 6

6 5 4

16 15 21

e

bl a r fe

s

an r t n

T

20 o n 19 a s a 18 h ) n3 ideฺ17 Saturday 7 v ฺ om t Gu c ฺ The Formula 1 n 2 e 85 s d t Due to the fact that there is a consistency associated with tuthe first day of the month affecting the date of the spfollowing S 1 and 3 Monday's (or any week day), the formula can be derived: @ s hi + (2 - [1 first day of week for anday ofe tmonth] r 3 Monday = ฺ[1 t the month] onMOD t7)o u+ s21 o ... and utilizing(m syntax se ... n MySQL n a r e mysql> SELECT '2008-02-01' + INTERVAL ((2 - DAYOFWEEK('2008-02-01') % 7) + 21) DAY T Third_Monday; lic t > AS e y u+--------------+ g hi N | Third_Monday | st

rd

rd

st

st

+--------------+ | 2008-02-18 | +--------------+

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-28

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Further Practice 3-C

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this further practice, you will utilize the fundamentals of searching date and text fields that were presented in this chapter.

Thi

1.

2. 3. 4.

Install a copy of the sakila database into a database with the same name. This is a sample database provided by MySQL and found in the Example Databases section of the http://dev.mysql.com/doc/ page. This database contains data associated with a madeup video rental store. If assistance is needed in installing this database, please contact your instructor for specific directions. Using the sakila database, review the structure of the customer, film, inventory and rental tables. Using the rental table, determine the maximum rental days where a customer checked out a video and returned it. In addition, include the average length of days that customers (who have returned the videos) have kept the videos. Create a report that identifies videos that are currently out. Include in the report the customers name and contact info, the video that they have checked out and the number of days they have had the movie checked out (from the current date). Note: Due to this data being slightly outdated, the days checked out will be quite lengthy.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-29

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

3.10 Summary In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

86

Thi

•

Search for a string in multiple columns

•

Compare case sensitivity for text and binary searches

•

Utilize INET_ATON and INET_NTOA functions

•

Convert string dates to SQL dates

•

Convert dates to integer values

•

Convert a date to a string

•

Search for a non-specific date

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

3-30

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 4

e

s

bl a r fe

an r t n

no IMPROVING INSERTS a as

uy g N

h ฺ ) n ฺv uide m o ฺc nt G 1 2 e pts Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

4

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

4.1

Thi

Chapter 4: Improving Inserts

IMPROVING INSERTS Overview This chapter provides the knowledge and skills necessary to improve the response time of queries that are responsible for inserting data. At the completion of this chapter, you will be able to:

88

•

Describe the process that takes place when an INSERT is performed

•

Use multiple-row insert statements to store many rows with one SQL statement

•

Utilize the INSERT ... SELECT statement to add/modify rows of data from one or more tables

•

Improve data inserts by utilizing LOAD DATA INFILE

•

Speed up bulk inserts with MyISAM

•

Speed up bulk inserts with InnoDB

e

s

Utilize MySQL extensions to standard SQL to improve data inserts

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e •

bl a r fe

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-1

MySQL Developer Techniques

4.2

INSERT Process When an INSERT is performed against MySQL, there is a defined process that takes place:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

89

Thi

Chapter 4: Improving Inserts

•

Connection to the server is opened

•

Query sent to the server

•

Query is parsed

•

Row is inserted into table

•

Index is created and inserted into index structure

•

Connection to the server is closed

Each of these steps has an associated cost connected to it in the form of performance. In addition, there is an initial overhead cost that takes place to open tables for each concurrently running query.

Connect Send Parse Insert Index Disconnect

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-2

MySQL Developer Techniques

4.2.1

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

90

Thi

Chapter 4: Improving Inserts

Multiple Row Inserts When there is a need to insert multiple rows of data, consider using INSERT statements with multiple VALUES lists to minimize the cost overhead related to creating a connection, sending the query to the server, parsing the server, opening the table and then closing the table and connection. This method can improve the speed of inserts considerably over the single row inserts method. The following statement demonstrates how a multiple insert would look:

mysql> INSERT INTO ID_table (ID,Name) VALUES -> (1,'Max'),(2,'Pat'),(3,'Roland'),(4,'Jeff'),(5,'Sarah');

For the most part this is a very simple approach to improving performance when inserting multiple rows of data. However, with almost every process, there are issues that need to be addressed and multiple-row inserts or updates are no exception. The following is a list of characteristics of multiple row data manipulation statements: 91

• Multi-row statements must fully succeed - When a transaction has multiple rows being modified (like an INSERT statement adding multiple rows of in one transaction), all the components of the transaction must succeed for the individual transaction to complete. Consider the following statement:

Connect Send Parse Insert Index Send Parse Insert Index

e

bl a r fe

Send Parse

Insert ns a r t n- Index

no Disconnect a s a h mysql> INSERT INTO employees (employeeID) VALUES n) ideฺ v ฺ -> (10001),(10002),(10003),(10004),(10001),(10005); om t Gu c ฺ 1 inserts ewould n take place because the next to last 2the With a transactional storage engine, none of s d t tu (assuming that the employeeID field is a value would cause a duplicate value key sinpa primary S @ s primary key). For a non-transactional hi engine, the first four values would be inserted and ancauseeantstorage r the next to last valueฺtwould error preventing the last two values from being inserted (due to the execution us terminating after the error was encountered). on of thetostatement o (m- Multiple-row • Portability se statements are less portable and less likely to be understood by other n n a database systems. r e T lic t • Too big of inserts - It is important to consider the database administrator (and the database e guy system) when determining how much data should be committed at any given time. This is also

N

important when considering how much data should and can be rolled back at one time in relation to transactional storage engines.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-3

MySQL Developer Techniques

4.2.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

92

Chapter 4: Improving Inserts

INSERT ... SELECT With the INSERT ... SELECT syntax, it is possible to insert many rows into a table from one or many tables. Similar to the multiple row inserts, this process can be used to minimize the cost associated with connecting and disconnecting in the INSERT process. The SELECT statement used can be any kind of SELECT statement, including joins and subqueries. In this case, we have created a table like City and called it City2, then added all the column/row data from the original City table in to this new table;

mysql> INSERT INTO City2 SELECT * FROM City;

The resulting rows will be the same as those of the original City table. REPLACE ... SELECT

s

o

an s a h ) 4.2.3 LOAD DATA INFILE n ideฺ v ฺ u oa m G The LOAD DATA INFILE statement reads rows from text file into c ฺ t 1 wayento insert 93 an existing table. This method is a much more 2efficient s t data then using INSERT statements (either p individual torudmultiple row s S INSERT statements). In most cases @wheretthe s DATA INFILE iaLOAD n h a statement is used, the data was exported using SELECT ... INTO r ฺt is theuSQL se complement. n OUTFILE statement o which The following o o t shows an example of exporting data and then importing that same data (m nse into another table: n a ice r T l t e y gumysql> SELECT LastName, FirstName, SchoolID N -> FROM class_roster hi

T

an r t n

-> ->

WHERE ClassCode LIKE 'HIS%' INTO OUTFILE 'C:/tmp/HIS_students.txt';

e

bl a r fe

REPLACE is a MySQL extension to the SQL standard. It either inserts, or deletes and inserts. REPLACE works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is inserted. The REPLACE .. SELECT syntax can be used instead of the INSERT ... SELECT statement to either insert a new row or update an existing row (which would actually be a delete and followed by an insert).

Connect Send Parse Insert Insert Insert Insert Insert Insert Insert Index Disconnect

mysql> LOAD DATA INFILE 'C:/tmp/HIS_students.txt' -> INTO TABLE db.History;

The first statement uses the SELECT ... INTO OUTFILE to load the text file '\tmp\HIS_students.txt' with all students that are enrolled in history classes (HIS is located in the first three characters of the class code). The second statement loads the History table with the values that were stored in the HIS_students.txt file.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-4

MySQL Developer Techniques

Chapter 4: Improving Inserts

Importing tab delimited or comma separated files:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

94

Thi

To import a data file, containing tab delimited or comma separated table data, the LOAD DATA INFILE command can be used. Suppose there is a file called City.csv that contains the following data: "1", "Kabul", "AFG", "Kabol", "1780000" "2", "Qandahar", "AFG", "Qandahar", "237500" "3", "Herat", "AFG", "Herat", "186800" "4", "Mazar-e-Sharif", "AFG", "Balkh", "127800" "5", "Amsterdam", "NLD", "Noord-Holland", "731200" "6", "Rotterdam", "NLD", "Zuid-Holland", "593321" "7", "Haag", "NLD", "Zuid-Holland", "440900" "8", "Utrecht", "NLD", "Utrecht", "234323" "9", "Eindhoven", "NLD", "Noord-Brabant", "201843" The following LOAD DATA INFILE statement could be used to import this data: mysql> LOAD DATA INFILE 'C:/tmp/City.csv' INTO TABLE City -> FIELDS TERMINATED BY ',' -> ENCLOSED BY '"' -> LINES TERMINATED BY '\n';

e

bl a r fe

s

an r t n

no a s a h Files on local host n) ideฺ v ฺ By default, MySQL assumes that the file is located on the serverm statement begins with LOAD DATA u client host o host. IfttheGthe LOCAL INFILE rather than with LOAD DATA INFILE, theฺc file is read from on which the statement is 1from which n 2 e issued. In other words, LOCAL means local to the client host the statement is issued. s dover the network to the server.In this case, the client t its contents u psends t program reads the data file and s S @ s i n h tra se t ฺ n o to u o m n ( ense a r T lic t e guy

N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-5

MySQL Developer Techniques Skipping lines in the data files

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

95

To ignore the initial part of the data file, use the IGNORE n LINES clause, where n is an integer that indicates the number of input lines to skip. This clause commonly is used when a file begins with a row of column names rather than data values. Suppose the file called City.csv contains header lines to identify the data columns as shown below: Note: Data exported from the world.City table "ID","Name","CountryCode","District","Population" "1", "Kabul", "AFG", "Kabol", "1780000" "2", "Qandahar", "AFG", "Qandahar", "237500" "3", "Herat", "AFG", "Herat", "186800" "4", "Mazar-e-Sharif", "AFG", "Balkh", "127800" "5", "Amsterdam", "NLD", "Noord-Holland", "731200" "6", "Rotterdam", "NLD", "Zuid-Holland", "593321" "7", "Haag", "NLD", "Zuid-Holland", "440900" "8", "Utrecht", "NLD", "Utrecht", "234323" "9", "Eindhoven", "NLD", "Noord-Brabant", "201843" To skip the first two input lines, a statement might be written like this:

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ Skipping or Transforming Column Values ndata values read from the file before 2or1to transform e s It is possible to skip columns in the data file, d t tuby specifying user variables in the column list spare available inserting them into the table. These features S @ s i and the optional SET clause. n h a t r t e ฺ to ua user To assign an input data n s variable rather than to a table column, provide the name of a o column user variable in the o column list.to If the column is assigned to a user variable but nothing else is done with (meffectnisstoeignore the column rather than to insert it into the table. Or, by including a the variable, the n a SET clause, r expressions ce can be used to transform the value before inserting it. T i l t uyeThe following example will skip the ID (first) column along with the city name and district being mysql> -> -> -> ->

96

g

N Thi

Chapter 4: Improving Inserts

LOAD DATA INFILE 'C:/tmp/City.csv' INTO TABLE City FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 2 LINES;

concatenated together with a space inbetween:

mysql> LOAD DATA INFILE 'C:/City.txt' INTO TABLE City -> (@skip,@Name,CountryCode,@District,Population) -> SET name=CONCAT(@Name,' ',@District);

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-6

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

97

Thi

Chapter 4: Improving Inserts

LOAD DATA INFILE and Duplicate Rows When new rows are added to a table with the INSERT or REPLACE statement, there are means to control how new rows are handled containing values that duplicate unique key values already present in the table. An error can be allowed to occur, the new rows can be ignored, or the old rows can be replaced with the new ones. LOAD DATA INFILE affords the same types of control over duplicate rows by means of two modifier keywords (IGNORE or REPLACE). However, its duplicate-handling behavior differs slightly depending on whether the data file is on the server host or the client host, so the data file location must be taken into account. When loading a file that is located on the server host, LOAD DATA INFILE handles rows that contain duplicate unique keys as follows: •

By default, an input record that causes a duplicate-key violation results in an error and the rest of the data file is not loaded. Records processed up to that point are loaded into the table.

•

If the IGNORE keyword is specified following the filename, new records that cause duplicatekey violations are ignored and no error occurs. LOAD DATA INFILE processes the entire file, loads all records not containing duplicate keys, and discards the rest.

•

e

bl a r fe

s

an r t n

If the REPLACE keyword is specified after the filename, new records that cause duplicate-key violations replace any records already in the table that contain the duplicated key values. LOAD DATA INFILE processes the entire file and loads all its records into the table.

no a IGNORE and REPLACE are mutually exclusive. One or the other can be specified, s but not both. a h For data files located on the client host, duplicate unique key handling is similar,ฺ except that the default n) with v ฺ is to ignore records that contain duplicate keys rather than to terminate daneerror. That is, the default i m u o is as though the IGNORE modifier is specified. The reason isG that the client/server protocol does ฺc for this t 1 n not allow transfer of the data file from the client 2 host to the server to be interrupted after it has started, so s in the tmiddle. de t there is no convenient way to abort the operation u p s sS @ i KEY values n PRIMARY h a t r t e ฺ INFILE The biggest danger with LOAD us is the ability to replace or delete the primary key values. This is onDATA o o t definitely a danger and(must m be taken e into account when performing inserts or updates using this method. With storage s n engines that support foreign keys (such as InnoDB and Falcon), the foreign key can be created with options (such as n a r ce NULL, etc.) that handle any deletes or replacements of the primary or candidate keys. RESTRICT, CASCADE, SET T i l t This can yeminimize the danger associated with LOAD DATE INFILE but for those storage engines that do not support u g Nforeign keys, caution is still required.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-7

MySQL Developer Techniques

4.2.4

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

98

Thi

Chapter 4: Improving Inserts

Transactional Multiple Inserts With transactional storage engines, it is possible to execute multiple insert statements in a single transaction. This is usually performed to provide atomicity to the data (all the actions are performed or none of the actions are performed); however, multiple insert statements in a single transaction can have a positive effect on performance. The following SQL is an example of performing multiple insert statements in a single transaction:

mysql> mysql> mysql> mysql> mysql> mysql>

START TRANSACTION; INSERT INTO city (NULL,'Bryson City','USA','North Carolina', 7800); INSERT INTO city (NULL,'Virginia City','USA','Virginia',5467); INSERT INTO city (NULL,'Morgantown','USA','West Virginia',11435); INSERT INTO city (NULL,'Julian','USA','Pennsylvania',4356); COMMIT;

Combining multiple insert statements into a single transaction can be beneficial to performance up to a point. Not only is it more costly to rollback large transactions, anything over approximately 2 megabytes of data will degrade performance rather than improve performance. In addition, the size of the log_buffer must be considered when considering the size of a transaction.

e

bl a r fe

s

an r t n

no a In MySQL, autocommit is a setting that tells the server to either consider each SQL statement a single transaction or s toasdetermine a wait for the user (or system) to enter an explicit command (either COMMIT or ROLLBACK) when all the h ) ฺ n statements for the transaction have been completed. If the autocommit mode is enabled (SET AUTOCOMMIT=1), each ฺv starts dnewe connections with autocommit i m SQL statement forms a single transaction of its own. By default, MySQL u co nt G ฺ enabled. 1 2 However, signored. de for transactional storage engines it is either t In non-transactional storage engines, these settingsp are u t s S necessary to disable the autocommit setting@ (SET AUTOCOMMIT=0) or explicitly tell the server when a transaction s i n h should begin. The START TRANSACTION (or BEGIN) t statement overrides the autocommit setting and tells the server tracommand e ฺ s everything that follows, up to an explicit (either COMMIT or ROLLBACK), will form a single transaction. n o to u o (m nse n a r ce T i l t uye g N START TRANSACTION, is it necessary?

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-8

MySQL Developer Techniques

4.3

Storage Engine Specifics Most storage engines have their own specific processes that can be addressed to improve the performance of the operations performed against them. Two of the main storage engines; MyISAM and InnoDB are not exception. In the case of speeding up insert operations, these two storage engines will be addressed.

99

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 4: Improving Inserts

4.3.1

Speeding up Bulk Inserts with MyISAM One of the drawbacks of performing bulk inserts is the amount of time that takes place to accomplish the task. Even though bulk inserts tend to be faster than other means of inserting large amounts of data, there are still ways to fine-tune MySQL to improve the inherent performance gains from such an operation. One of these ways is to disable non-unique index keys during the bulk insert operation. The following discussion applies to bulk insert operations performed against tables utilizing the MyISAM storage engine. Disabling keys The process by which non-unique indexes are disabled with MyISAM tables is to alter the table that the operation will be performed against:

e

bl a r fe

s

mysql> ALTER TABLE table_name DISABLE KEYS;

an r t n

T

o n a Once this statement is executed, the server itself will ignore these indexes when the best s determining a query execution plans either by using SELECT or evaluating any statements with the EXPLAIN h n) ideฺ feature. v ฺ Enabling keys om t Gu c ฺ The process by which non-unique indexes are enabled n with MyISAM tables is to alter 21 (or dre-enabled) e s t the table that the operation will be performed against: sp s Stu @ i nENABLE KEYS; mysql> ALTER TABLE table_name h a t r t e s nฺ userver o o o Once this statement is executed, the itself will evaluate these indexes when determining the best t (mplansneither e by using SELECT s query execution or evaluating any statements with the EXPLAIN n a ice r feature. T l t e Delaying insert y 100 u g Bullk inserts can be improved by using the DELAYED option with INSERT. This is due to the client N i h not having to wait until the bulk insert is completed to execute other statements.

Rebuilding at one time The advantage of disabling keys and then re-enabling them after a bulk insert operation comes in the form of rebuilding the indexes once, rather than having the server performing a modification against the physical storage of the indexes after each individual operation. Even though the re-enabling operation requires the server to rebuild the indexes for the whole table, the time that it takes is much faster than if the keys were enabled during the bulk insert operation. If there is a desire to achieve any benefit from this process, it is best to perform these actions against an empty table. When should this be used? To gain benefit from this process, the rows being inserted must be significantly more than the rows already existing in the table. _________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-9

MySQL Developer Techniques

4.3.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

101

Chapter 4: Improving Inserts

Speeding up Bulk Inserts with InnoDB InnoDB Defaults InnoDB default options are very conservative and in several situations increasing the buffers can have a high impact on the performance. To speed up inserts on an InnoDB-only setup, the rule of a thumb is to set the innodb_buffer_pool size to 50-80% of the computer's memory and to set innodb_log_file_size to about 25% of the innodb_buffer_pool_size. If the MySQL server is running on several threads, which all work on the InnoDB table(s), performance can be gained by setting the innodb_flush_log_at_trx_commit value to 0. This adds some risk of losing some transactions in the case of a power failure or an unclean mysqld shutdown, but it can improve speed. Primary key order inserts

Please note that when inserting several rows at once in InnoDB table, it is fastest, if the rows are inserted in primary key order. This is because of the clustered index, which InnoDB always uses. 102

Disable AUTOCOMMIT

s

an r t n

When importing data into InnoDB, make sure that the MySQL server does not have autocommit mode enabled because that requires a log flush to disk for every insert. To disable autocommit during the import operation, surround it with SET AUTOCOMMIT and COMMIT statements:

no a s a h mysql> SET AUTOCOMMIT=0; n) ideฺ v ... SQL import statements ... ฺ mysql> COMMIT; om t Gu c ฺ 1 are fasterento import into an InnoDB table, even 2files s If using the mysqldump option --opt, dump d t p SandtuCOMMIT s without wrapping them with the SET AUTOCOMMIT statements. @ s i n h Turn off UNIQUE_CHECKS ra t ฺt onusecondary 103 se keys, table import performance can be improved by n If there are UNIQUE o constraints ooff the uniqueness to checks during the import session: temporarily turning m ( e n ens aUNIQUE_CHECKS=0; r mysql> SET T lic t ... e import operation ... y SET UNIQUE_CHECKS=1; umysql> g hi N

T

104

e

bl a r fe

For big tables, this saves a lot of disk I/O because InnoDB can use its insert buffer to write secondary index records in a batch. Be certain that the data contains no duplicate keys. UNIQUE_CHECKS allows but does not require storage engines to ignore duplicate keys. Turn off FOREIGN_KEY_CHECKS

If there are FOREIGN KEY constraints in the InnoDB tables, table import performance can be improved by turning foreign key checks off for the duration of the import session: mysql> SET FOREIGN_KEY_CHECKS=0; ... import operation ... mysql> SET FOREIGN_KEY_CHECKS=1;

For big tables, this can save a lot of disk I/O. _________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-10

MySQL Developer Techniques

4.4

Chapter 4: Improving Inserts

MySQL Extensions The MySQL server contains additions to the SQL standard when it comes to functions associated with inserts.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

105

•

INSERT ... SELECT - This statement syntax can quickly insert multiple rows into a table from one or more tables. The following are some of the additional features of this statement that should be understood: o

INSERT IGNORE ... SELECT - The IGNORE clause will cause the statement to ignore rows that would cause duplicate-key violations.

o

Same table - The target table of the INSERT statement may appear in the FROM clause of the SELECT part of the query. In this case, MySQL creates a temporary table to hold the rows from the SELECT and then inserts those rows into the target table. However, the table referenced can not be a TEMPORARY table itself because TEMPORARY tables cannot be referred to twice in the same statement.

e

T

bl a r • INSERT DELAYED - The DELAYED option for the INSERT statement is a MySQL extension fe s n to standard SQL that is very useful if there are transactions that cannot or need not wait for the a there trand INSERT to complete. This is a common situation when MySQL is used for logging n are also periodically run SELECT and UPDATE statements that take a long notime to complete. a However, the negative side to this syntax is it is not transactionally s safe. The end user is never a given any response to know how long its going to take for the INSERT to happen. If the server h ) ฺ n crashes, or is forcefully shutdown, the delayed INSERT's will be lost. ฺv uide m o • INSERT ... ON DUPLICATE KEY UPDATE - ThisG statement syntax allows a user to ฺc if there t 106 1 n perform an update against the data being2entered is already an identical value in either a e s tuddemonstrates t UNIQUE index or PRIMARY KEY. The following how this statement syntax will p s sS look in action: @ i n h a t r t e s nฺ (ID,Name,Old_Name) mysql> INSERT INTO ID_table VALUES (1,'John','') uOld_Name=Name,Name='John'; o o o -> ON DUPLICATE KEY UPDATE t (m nse n a r This statement ce will perform an INSERT into the ID_table with a value of 1 for the ID T i l t column 'John' for the Name column, and an empty string for the Old_Name column if a row uye g in the table does not already have an ID column with the value of 1. If there is a row with an hi N ID column with a value of 1 already in the table, then the following action would be performed:

mysql> UPDATE ID_table SET Old_Name=Name,Name='John' WHERE ID=1;

This syntax (INSERT ... ON DUPLICATE KEY UPDATE) eliminates the need to do two round trips to the database of an insert or update which ultimately improves performance.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-11

MySQL Developer Techniques •

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

107

Thi

Chapter 4: Improving Inserts

REPLACE - This statement syntax works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is inserted. The following demonstrates how this statement syntax will look in action:

mysql> REPLACE ID_table (ID,Name) VALUES (1,'John');

This statement will perform an INSERT into the ID_table with a value of 1 for the ID column and 'John' for the Name column if a row in the table does not already have an ID column with the value of 1. If there is a row with an ID column with a value of 1 already in the table, then the following actions would be performed: mysql> DELETE FROM ID_table WHERE ID=1;

e

bl a r fe

mysql> INSERT INTO ID_table (ID,Name) VALUES (1,'John');

s

an r t n

This syntax (REPLACE) can save round trips to the database which ultimately improves performance.

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-12

MySQL Developer Techniques

Chapter 4: Improving Inserts

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Lab 4-A

Thi

In the following lab, large inserts will be performed against InnoDB tables with referential integrity checking turned on. Then, the same operations will take place with the referential integrity checking turned off to evaluate the amount of difference performing large insert operations between the two processes. 1.

Action (you perform). Alter the existing world database tables to utilize the InnoDB storage engine:

mysql> ALTER TABLE Country ENGINE=InnoDB; mysql> ALTER TABLE CountryLanguage ENGINE=InnoDB; mysql> ALTER TABLE City ENGINE=InnoDB;

e

bl a r fe

s

Response: Each of the statements returns a Query OK response with the number of rows of data that was affected by the change. These tables will need to be InnoDB before we can add foreign keys to them (InnoDB is the most popular MySQL transactional storage engine) in steps to follow.

an r t n

no a s a h n) ideฺ v ฺ mysql> CREATE TABLE City_copy LIKE City; om t Gu c ฺ n e s21thetstructure d t Response: A new table is created thatp contains u of the City table but with none of the data of the table. This table will@ be s used to test S the inserts of large amounts of data to determine the is checks are turned on and turned off. n difference in performance a when referential integrity h t r ฺtAdd foreign sekey constraints to the original world tables that were altered in n 3. Action (you perform). u o o to step 1: (m e an icens r T mysql>t ALTER TABLE l City_copy ADD FOREIGN KEY (CountryCode) REFERENCES Country (Code); e y gumysql> ALTER TABLE Country ADD FOREIGN KEY (Capital) REFERENCES City_copy (Id); 2.

Action (you perform). Create a copy of the City table structure to a new table called City_copy:

N

Response: Each of the statements returns a Query OK response with the number of rows of data that was affected by the change. Notes: These tables now have foreign key constraints that will be checked.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-13

MySQL Developer Techniques 4.

Chapter 4: Improving Inserts

Action (you perform). Insert the contents from the City table into the City_copy table:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

mysql> INSERT INTO City_copy SELECT * FROM City;

Thi

Response: The City_copy table now contains the same number of rows as the City table. How long did it take for the City_copy table to be populated with the City data? _________ seconds. 5.

Action (you perform). Insert the contents of the City_copy table back into itself a total of five times by executing the following SQL:

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population -> FROM City_copy;

Response (1): The City_copy table now contains double the number of rows that it originally contained. How long did it take for the City_copy table to insert the contents of itself (approximately 4,000 records)? _________ seconds.

e

bl a r fe

s

an r t n

Note: The NULL in the SELECT statement is due to the first column being an auto_increment primary key column. The NULL allows the column to utilize the auto_increment feature which ensures that the contents of that column will be unique.

no a s a h mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, n) ideฺ Population -> FROM City_copy; v ฺ om t Gu c ฺ 1 approximately n four times the number of rows Response (2): The City_copy table now2 contains e s d t that it originally contained. How longp did it take for tuthe City_copy table to insert the contents of s _________ S itself (approximately 8,000 records)? seconds. @ s i n h a t r t e nฺ SELECT mysql> INSERT INTO City_copy us NULL, Name, CountryCode, District, Population o o o -> FROM City_copy; t (m nse n a r ceThe City_copy table now contains approximately eight times the number of rows T i l Response (3): t e it originally contained. How long did it take for the City_copy table to insert the contents of guy that itself (approximately 16,000 records)? _________ seconds.

N

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population -> FROM City_copy;

Response (4): The City_copy table now contains approximately sixteen times the number of rows that it originally contained. How long did it take for the City_copy table to insert the contents of itself (approximately 32,000 records)? _________ seconds.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-14

MySQL Developer Techniques

Chapter 4: Improving Inserts

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population -> FROM City_copy;

Thi

Response (5): The City_copy table now contains approximately thirty two times the number of rows that it originally contained. How long did it take for the City_copy table to insert the contents of itself (approximately 65,000 records)? _________ seconds. mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population -> FROM City_copy;

Response (6): The City_copy table now contains approximately sixty four times the number of rows that it originally contained. How long did it take for the City_copy table to insert the contents of itself (approximately 130,000 records)? _________ seconds. 6.

e

bl a r fe

Action (you perform). Remove the contents from the City_copy table:

mysql> TRUNCATE City_copy;

s

an r t n

no a s a h ) ninto eฺ v 7. Action (you perform). Insert the contents from the City ฺtable the City_copy table: d i m u o ฺc tG 1 mysql> INSERT INTO City_copy SELECT * FROM City; en 2 pts Stud s @nowtcontains is the same number of rows as the City table. How Response: The City_copyntable h a r t long did it take for the ฺCity_copy setable to be populated with the City data? _________ seconds. n u o o Turn 8. Action (you perform). tooff the foreign key constraints: m ( e n ens aFOREIGN_KEY_CHECKS=0; r mysql> SET T lic t e guy Response: This will affect any actions against the contents of the InnoDB tables that are

Response: The data contained within the City_copy table has been deleted but the structure of the City_copy table remains the same. This will provide us with the ability to perform a comparable test with the foreign key constraints turned off for the remainder of this lab.

N

manipulated during the current session.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-15

MySQL Developer Techniques 9.

Chapter 4: Improving Inserts

Action (you perform). With the foreign key constraints turned off, insert the contents of the City_copy table back into itself a total of five times by executing the following SQL:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population -> FROM City_copy;

Thi

Response (1): The City_copy table now contains double the number of rows that it originally contained. How long did it take for the City_copy table to insert the contents of itself (approximately 4,000 records)? _________ seconds. Note: The NULL in the SELECT statement is due to the first column being an auto_increment primary key column. The NULL allows the column to utilize the auto_increment feature which ensures that the contents of that column will be unique.

e

bl a r fe

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population -> FROM City_copy;

s

an r t n

Response (2): The City_copy table now contains approximately four times the number of rows that it originally contained. How long did it take for the City_copy table to insert the contents of itself (approximately 8,000 records)? _________ seconds.

no a s a h mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, n) iDistrict, eฺ Population v ฺ -> FROM City_copy; d om t Gu c ฺ 1 approximately n eight times the number of rows 2contains e Response (3): The City_copy table now s d t that it originally contained. How long spdid itstakeSfortuthe City_copy table to insert the contents of @ itself (approximately 16,000 n records)? _________ i seconds. h a t r t e us NULL, Name, CountryCode, District, Population onฺ toSELECT mysql> INSERT INTO o City_copy -> FROM City_copy; (m nse n a r ceThe City_copy table now contains approximately sixteen times the number of T i l t Response (4): e guy rows that it originally contained. How long did it take for the City_copy table to insert the

N

contents of itself (approximately 32,000 records)? _________ seconds.

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population -> FROM City_copy;

Response (5): The City_copy table now contains approximately thirty two times the number of rows that it originally contained. How long did it take for the City_copy table to insert the contents of itself (approximately 65,000 records)? _________ seconds.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-16

MySQL Developer Techniques

Chapter 4: Improving Inserts

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population -> FROM City_copy;

Thi

Response (6): The City_copy table now contains approximately sixty four times the number of rows that it originally contained. How long did it take for the City_copy table to insert the contents of itself (approximately 130,000 records)? _________ seconds. 10. Action (you perform). Evaluate the amount of time that each insert took in step 9 against each associated insert in step 5. Which step consistently had a better response time and why? ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-17

MySQL Developer Techniques

4.5

Summary In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

108

Thi

Chapter 4: Improving Inserts

•

Describe the process that takes place when an INSERT is performed

•

Use multiple-row insert statements to store many rows with one SQL statement

•

Utilize the INSERT ... SELECT statement to add/modify rows of data from one or more tables

•

Improve data inserts by utilizing LOAD DATA INFILE

•

Speed up bulk inserts with MyISAM

•

Speed up bulk inserts with InnoDB

•

Utilize MySQL extensions to standard SQL to improve data inserts

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

4-18

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 5

e

s

bl a r fe

an r t n

COMPLEX as a no h ฺ ) n CALCULATIONS ฺv uide om

uy g N

ฺc nt G 1 2 e pts Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

5

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

5.1

Thi

Chapter 5:Complex Calculations

COMPLEX CALCULATIONS Overview This chapter provides the knowledge and skills necessary to perform simple to complex calculations using SQL. At the completion of this chapter, you will be able to:

110

•

Simulate an aggregate multiplication function

•

Use variables to create a running total query

•

Utilize COALESCE to avoid divide by zero errors

•

Calculate the median value

•

Simulate the SQL RANK function in MySQL

•

Utilize SQL to solve complex problems

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-1

MySQL Developer Techniques

5.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

111

Thi

Chapter 5:Complex Calculations

Aggregate Multiplication MySQL provides multiple aggregate functions that can be used to compute a single result value from a collection of records. However, MySQL does not provide a means to create an aggregate function for multiplication purposes. This limitation can be corrected with some creative SQL. Using logarithms The following formula shows that a list of logarithms that are added together is equivalent to the logarithms of that list being multiplied together: log(x) + log(y) + log(z) = log(x*y*z) When the opposite function EXP is used, the resulting formula would look like this: exp(log(x) + log(y) + log(z)) = x*y*z This process can then be used in aggregate functions.

112

e

bl a r fe

How can this be used? The following chart shows the rate of return for a publicly traded company over a 10 year span: YEAR 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

ROR* 44% -25% 8% -2% 35% 57% 1% 11% -8% -21%

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e * Rate of return

uy With an initial investment of $1,000 in 1997, the current value of the investment of 2007 would need to g be calculated. The first thought may be to average the rate of return and then multiply this by the initial N $1,000 investment:

mysql> SELECT AVERAGE(ROR) FROM investment;

This would result in an average return of approximately 9.9% over the 10 year time span. However, this does not take into account compound interest.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

113

Thi

Chapter 5:Complex Calculations

Calculating compound interest The first step in calculating compound interest is to obtain the logarithm of the rate of return for each year: mysql> SELECT YEAR, 1+ROR AS multiplier, LOG(1+ROR) FROM investment; +------+------------+--------------------+ | YEAR | multiplier | LOG(1+ROR) | +------+------------+--------------------+ | 1998 | 1.44 | 0.36464311358791 | | 1999 | 0.75 | -0.28768207245178 | | 2000 | 1.08 | 0.076961041136128 | | 2001 | 0.98 | -0.020202707317519 | | 2002 | 1.35 | 0.30010459245034 | | 2003 | 1.57 | 0.45107561936022 | | 2004 | 1.01 | 0.0099503308531681 | | 2005 | 1.11 | 0.10436001532424 | | 2006 | 0.92 | -0.083381608939051 | | 2007 | 0.79 | -0.23572233352107 | +------+------------+--------------------+ 10 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a 114 s a h n) ideฺ v mysql> SELECT SUM(LOG(1+ROR)) FROM investment; ฺ +------------------+ om t Gu c ฺ | SUM(LOG(1+ROR)) | 21 den +------------------+ s t | 0.68010599048258 | sp s Stu +------------------+ @ i n h 1 row in set (#.## sec) a t r t e onฺ to us o This of course(only m showssethe logarithmic results. The next step is to use the EXP function to turn the n logarithmic results intonthe multiplier that determines the return percentage with compound interest: a r e c T i l et SELECT EXP(SUM(LOG(1+ROR))) FROM investment; y mysql> u Ng+----------------------+ With the formula created and tested for the logarithm that is going to be used, the next step is to put this formula into an aggregate function that adds all the logarithmic results together:

| EXP(SUM(LOG(1+ROR))) | +----------------------+ | 1.9740869555715 | +----------------------+ 1 row in set (#.## sec)

This percentage, 97% and some change, is quite different than the average of 9.9% calculated by just averaging the rate of returns.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-3

MySQL Developer Techniques The final step is to then take this value and multiply it by the initial investment:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

115

Thi

Chapter 5:Complex Calculations

mysql> SELECT EXP(SUM(LOG(1+ROR)))*1000 FROM investment; +---------------------------+ | EXP(SUM(LOG(1+ROR)))*1000 | +---------------------------+ | 1974.0869555715 | +---------------------------+ 1 row in set (#.## sec)

Over ten years, even including the bad years, the initial investment of $1,000 nearly doubled. Of course, the purpose of this example was not to show how much a person can make (or lose) in stocks, but to show how creative SQL can be used to solve perceived limitations in the language.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-4

MySQL Developer Techniques

5.3

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

116

Thi

Chapter 5:Complex Calculations

Running Total Query In the course of almost every analytical presentation, someone will inevitably request that a running total be presented. In the case of the aggregate multiplication example shown in the last section, an analytical minded person may request to see a running total showing how the investment looked at the end of each year. With external software, this could be accomplished with some creative functions to look something like the following: YEAR 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

ROR 44% -25% 8% -2% 35% 57% 1% 11% -8% -21%

EOY* $1,440.00 $1,080.00 $1,166.40 $1,143.07 $1,543.15 $2,422.74 $2,446.97 $2,716.14 $2,498.84 $1,974.09

e

bl a r fe

s

an r t n

no a s a h Creative SQL n) idcould eฺ create a running total v Sure, using an external application and some number crunching a presenter ฺ ulet MySQL handle it all. The like the one displayed above, but why not just be creative with SQLG and om c ฺ t following SQL example uses a self-join on the investment to ncreate the running total requested: 21 table e s d t sp s Stu mysql> SELECT a.YEAR, a.ROR, EXP(SUM(LOG(1+b.ROR)))*1000 AS balance @ i -> FROM investment a JOIN n investment b ON (a.YEAR >= b.YEAR) h -> GROUP BY a.YEAR, a.ROR; tra se t ฺ n +------+-------+-----------------+ o to u| o | YEAR | ROR |m balance +------+-------+-----------------+ se n |( 1439.9999976158 n a | 1998 | r0.44 | e c T i l t | 1999 | -0.25 | 1079.9999982119 | e | y 2000 | 0.08 | 1166.3999961376 | u g| 2001 | -0.02 | 1143.0719967363 | * End of year

N

| 2002 | 0.35 | 1543.1471887807 | | 2003 | 0.57 | 2422.7410753483 | | 2004 | 0.01 | 2446.9684855603 | | 2005 | 0.11 | 2716.1350175134 | | 2006 | -0.08 | 2498.8442209692 | | 2007 | -0.21 | 1974.0869509493 | +------+-------+-----------------+ 10 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-5

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

117

Thi

Chapter 5:Complex Calculations

Looking at how this works For the most part, the self join is straight forward, but what is actually going on to provide the results that is being requested? To see what is happening, the following SQL removes the SUM function and the corresponding GROUP BY to show the contents that make up the aggregate results: mysql> SELECT a.YEAR, a.ROR, EXP((LOG(1+b.ROR)))*1000 AS balance -> FROM investment a JOIN investment b ON (a.YEAR >= b.YEAR) -> ORDER BY a.YEAR, a.ROR; +------+-------+------------------+ | YEAR | ROR | balance | +------+-------+------------------+ | 1998 | 0.44 | 1439.99999761581 | | 1999 | -0.25 | 1439.99999761581 | | 1999 | -0.25 | 750 | | 2000 | 0.08 | 1439.99999761581 | | 2000 | 0.08 | 750 | | 2000 | 0.08 | 1079.99999821186 | | 2001 | -0.02 | 1439.99999761581 | | 2001 | -0.02 | 750 | | 2001 | -0.02 | 1079.99999821186 | | 2001 | -0.02 | 980.000000447035 | ... | 2007 | -0.21 | 980.000000447035 | | 2007 | -0.21 | 1349.99999403954 | | 2007 | -0.21 | 1569.99999284744 | | 2007 | -0.21 | 1009.99999977648 | | 2007 | -0.21 | 1109.99999940395 | | 2007 | -0.21 | 920.000001788139 | | 2007 | -0.21 | 790.000006556511 | | 2007 | -0.21 | 1439.99999761581 | | 2007 | -0.21 | 750 | | 2007 | -0.21 | 1079.99999821186 | +------+-------+------------------+ 55 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a The resulting one row for the first year, two rows for the second year and so on. This is r query ceproduces T i l t due to the first year resulting in one row that meets the join condition (a.YEAR >= b.YEAR), the e

uy second year resulting in two rows that meet the join condition, and so on. The values (represented by the g N balance column) are then aggregated together to produce the running total that is displayed when the SUM and GROUP BY clauses are used.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-6

MySQL Developer Techniques

5.4

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

118

Chapter 5:Complex Calculations

Avoiding Divide by Zero Divide by zero is a problem in computational equations and must be dealt with when developing any mathematical equation, either in SQL or any other programming language. When MySQL faces a divide-by-zero error, it fails silently and produces a NULL response.

mysql> SELECT 50/0 AS response +----------+ | response | +----------+ | NULL | +----------+ 1 row in set (#.## sec)

This of course may not be the best way to handle a divide-by-zero error and using creative SQL to avoid divide-by-zero errors should be implemented. Feeding a NULL into an equation The COALESCE function is an ANSI standard function that takes one to numerous arguments and provides the first non-NULL argument in response. There is an additional function called ISNULL that acts similarly to COALESCE but is limited in that it only takes two arguments, in not an ANSI standard function and most importantly takes on the datatype of first argument entered. COALESCE takes on the datatype of the response (or first non-NULL value) which can eliminate potential future problems. The following demonstrates the use of the COALESCE function:

e

bl a r fe

s

an r t n

T

no a s a h n) ideฺ v ฺ mysql> SELECT COALESCE(NULL, NULL, NULL, 0) AS response; om t Gu c ฺ +----------+ 21 den | response | s t +----------+ sp s Stu | 0 | @ i n h +----------+ a t r t e 1 row in set (#.## sec)nฺ o to us o (mfunction e itself does not solve the divide-by-zero error, but is the first step. The The COALESCE by sNULLIF n n a second step is using the function. The NULLIF function is an ANSI standard function that r e 119 c If both arguments T i l t takes two arguments. are identical, the function returns a NULL. If the two earguments are not identical the first value is returned. y u g N i h mysql> SELECT NULLIF(10,10) AS response; +----------+ | response | +----------+ | NULL | +----------+ 1 row in set (#.## sec) mysql> SELECT NULLIF(10,0) AS response; +----------+ | response | +----------+ | 10 | +----------+ 1 row in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-7

MySQL Developer Techniques

The final step to providing a solution to divide-by-zero errors is to combine the COALESCE and NULLIF functions. The following produces a numeric zero when a divide-by-zero would otherwise cause a problem:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

120

Thi

Chapter 5:Complex Calculations

mysql> SET @INPUT1 = 0; Query OK, 0 rows affected (#.## sec) mysql> SELECT COALESCE(50/NULLIF(@INPUT1,0),0) AS response; +----------+ | response | +----------+ | 0.0000 | +----------+ 1 row in set (#.## sec)

The response produced is a numeric zero because the statement 50/NULLIF(@INPUT1,0) will produce a NULL output thus causing the COALESCE function to ignore that first response and go with the next one: 0. However, if @INPUT1 is set to a numeric value that is not zero (thus would not produce a divide-by-zero error), the first value of the COALESCE would not be a NULL and become the response provided.

e

bl a r fe

s

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

mysql> SET @INPUT1 = 5; Query OK, 0 rows affected (#.## sec) mysql> SELECT COALESCE(50/NULLIF(@INPUT1,0),0) AS response; +----------+ | response | +----------+ | 10.0000 | +----------+ 1 row in set (#.## sec)

an r t n

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-8

MySQL Developer Techniques

5.5

Calculate the Median Value In many cases the average value of a set of data is not a consistent sampling of the data. Wide ranges of values can skew average results and provide an unrealistic assessment of data. Consider the following chart representing the continents and their populations:

121

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 5:Complex Calculations

Name

Population

Asia

3,705,025,700

Africa

784,475,000

Europe

730,074,600

North America

482,993,000

South America

345,780,000

Oceania

30,401,150

Antarctica

0

e

bl a r fe

s

an r t n

Based on the data in this chart, the average population for the continents is 868,392,779. A quick glance at the data reveals this is not accurate. Asia is so much larger in population than the other continents along with Antarctica having no population. This results in a skewed outcome and not representative of the data. A more accurate calculation would be the median value.

g

no a s a h Calculating the median value n) ideฺ v ฺ The median value is calculated by using the middle valueoofm a data set. Ifuthere was a grouping of values G c ฺ t that consisted of 1,3,5,20,99, the median value would 5. In the the population chart, the single ncaseIfof there 21ofbe 482,993,000. e s d middle value is North America with a population was an even number of t p Stu located in the center continents, this would require averagingsthe two populations of the data. Easy @ s i enough to determine when doing it by hand, but how is this done with SQL. n h ฺtra use t n Multiple paths to theo median o o to accomplish the same task. Some are more efficient than others, tways m In MySQL there are multiple ( e nsto create. Either way, in MySQL determining the median value can be while rothers an areiceasier e T accomplished inl one of three ways: t e y u

N Thi

•

Creating an implicit temporary table

•

Creating a stored procedure

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-9

MySQL Developer Techniques

5.5.1

Median Value via Implicit Temporary Tables Using an SQL user variable and nested SELECT can provide creative solutions to perceived limitations of SQL. In the following SQL statements, a user variable called @counter is created for use in a nested SELECT statement:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

122

Thi

Chapter 5:Complex Calculations

mysql> SET @counter := 0; Query OK, 0 rows affected (0.00 sec) mysql> SELECT FLOOR(AVG(table_a.Population)) AS median_pop -> FROM (SELECT @counter := @counter+1 AS Count, -> Population -> FROM Continents ORDER BY Population) AS table_a, -> (SELECT COUNT(*) AS totalrows FROM Continents) AS table_b -> WHERE Count IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1); +------------+ | median_pop | +------------+ | 482993000 | +------------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

123

no a s a h n)the values • AVG(value) - The first portion AVG(value) will v take eฺthat are returned and ฺ d i average them together. If there are an odd numberm theu values returned would be the o of rows, G c same values so averaging them would simply provide the one value in response. If there are ฺ t n 21 would e an even number of rows, the valuests returned be the middle two values which would p Stud then be averaged together. s @ is n h • (SELECT @counter := @counter+1 AS Count, Population FROM a t r t BY sPopulation) e ฺ Continents n ORDER AS table_a - This portion of the script acts o to u that is being used in the SQL statement. This table acts as the as the firsto table (table_a) m ( table nwhich counter sewill produce a value from 1 to 7 based on the values in the Continents n a r e table. ic l et T

uy g N

For the most part the SQL is pretty standard; however, the following explains portions of the SQL that may seem a little awkward:

•

(SELECT COUNT(*) AS totalrows FROM Continents) AS table_b - This portion of the script acts as the second table (table_b) that is being used in the SQL. The only column in this table, called totalrows, will contain the number of rows from the incomes table. This totalrows column will be used to determine the middle rows which is necessary for determining the median value.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-10

MySQL Developer Techniques •

Chapter 5:Complex Calculations

WHERE counter IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1) - The

totalrows column that is obtained from the second table (table_b), is used here to determine the middle rows.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

o

o

Odd number of rows - When there are an odd number of rows, the formula FLOOR((totalrows+1)/2) will evaluate to (totalrows+1)/2. The second formula FLOOR(totalrows/2)+1 will also evaluate to (totalrows+1)/2. Thus providing two values that are identical to the AVG(value) aggregate function. In a list that contains 1, 2, 3, 4, 5 the following results would exist: ƒ

FLOOR((totalrows+1)/2) => FLOOR((5+1)/2) => 3

ƒ

FLOOR(totalrows/2)+1 => FLOOR(5/2)+1 => 3

Even number of rows - Where there are an even number of rows, the formula FLOOR((totalrows+1)/2) will evaluate to totalrows/2. The second formula FLOOR(totalrows/2)+1 will evaluate to (position/2+1). This results in the two values returned being the values from the two middle rows in the list. In a list that contains 1, 2, 3, 4, 5, 6 the following results would exist:

s

an r t n

ƒ

FLOOR((totalrows+1)/2) => FLOOR((6+1)/2) => 3

ƒ

FLOOR(totalrows/2)+1 => FLOOR(6/2)+1 => 4

e

bl a r fe

no a s a h n) ideฺ FLOOR() function ฺv om Guof the numeric value that is supplied. The SQL FLOOR() function returns the largest integer that1isฺc less than ortequal n 2value thatdisesupplied. This function will return the same data type of the numeric s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t uye g hi N

T

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-11

MySQL Developer Techniques

Chapter 5:Complex Calculations

5.5.1.1 Evaluating Performance

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

124

The SQL statement that was used to determine the median value for the Continents table produced the correct output but at what cost to performance. Utilizing the EXPLAIN function can provide clues to performance issues: mysql> EXPLAIN -> SELECT FLOOR(AVG(table_a.Population)) AS median_pop -> FROM (SELECT @counter := @counter+1 AS Count, Population -> FROM Continents ORDER BY Population) -> AS table_a, -> (SELECT COUNT(*) AS totalrows FROM Continents) -> AS table_b -> WHERE Count IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1); +----+---------+------------+----+----+-------+----+----+------+------------------------------+ | id | selec.. | table | .. | .. | key | .. | .. | rows | Extra | +----+---------+------------+----+----+-------+----+----+------+------------------------------+ | 1 | PRIMARY | | .. | .. | NULL | .. | .. | 1 | | | 1 | PRIMARY | | .. | .. | NULL | .. | .. | 7 | Using where | | 3 | DERIVED | Continents | .. | .. | Pop.. | .. | .. | 7 | Select tables optimized away | | 2 | DERIVED | Continents | .. | .. | Pop.. | .. | .. | 7 | Using index | +----+---------+------------+----+----+-------+----+----+------+------------------------------+ 4 rows in set (0.00 sec)

e

bl a r fe

s

an r t n

T

no a s a h The first nested SELECT statement (id #2) is using the Continents.Population n) ideฺ index, but the v ฺ second nested SELECT statement (id #3) is able to completely uthe SELECT statement due to om remove G c the MyISAM storage engine having an internal row count. ฺ t n 21to attempt e s d t Other storage engines (such as InnoDB) would have to utilize an index. With the SELECT 125 spbe ables toSbetuused when statement as it currently is, no index@ would requesting a count on all columns: i n COUNT(*). To overcome thisalimitation intnon-MyISAM storage engines, an indexed column can be h tr sfeatures: e ฺ used instead of * to utilize optimization n u o o to m ( e an icens r T l t e y gu N hi Note: Some of the content from this output has been removed for printing purposes to ensure that the more important columns for this discussion are displayed.

mysql> EXPLAIN -> SELECT FLOOR(AVG(table_a.Population)) AS median_pop -> FROM (SELECT @counter := @counter+1 AS Count, -> Population -> FROM Continents ORDER BY Population) AS table_a, -> (SELECT COUNT(Population) AS totalrows FROM Continents) AS table_b -> WHERE Count IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1); +----+---------+------------+----+----+------------+----+----+------+-------------+ | id | selec.. | table | .. | .. | key | .. | .. | rows | Extra | +----+---------+------------+----+----+------------+----+----+------+-------------+ | 1 | PRIMARY | | .. | .. | NULL | .. | .. | 1 | | | 1 | PRIMARY | | .. | .. | NULL | .. | .. | 7 | Using where | | 3 | DERIVED | Continents | .. | .. | Population | .. | .. | 7 | Using index | | 2 | DERIVED | Continents | .. | .. | Population | .. | .. | 7 | Using index | +----+---------+------------+----+----+------------+----+----+------+-------------+ 4 rows in set (#.## sec)

Using the Continents.Population index in the COUNT function has allowed the second table (id #3) to now use the index and thus improves the performance of the SELECT statement.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-12

MySQL Developer Techniques

5.5.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

126

Thi

Chapter 5:Complex Calculations

Creating a Median Stored Procedure The MySQL stored programming language is a block-structured programming language based on the ANSI SQL:2003 SQL/PSM (Persistent Stored Module) specification. This language includes many of the common programming tools expected in a block-structured programming language such as manipulating variables, managing execution through conditions, repeated routine processing along with handling errors. In the course of using MySQL, a developer may find it easier to produce a stored procedure to eliminate external coding or repetition of tasks. Thus, if there is a need to obtain the median value on a regular basis against datasets, it may make sense to create a stored procedure to handle the task. A median procedure Using the process shown for obtaining a median value using implicit temporary tables can be moved easily into a stored procedure that can be reused over and over again. The following is the median stored procedure in its entirety which will be followed up with a detailed look at how it works:

mysql> DELIMITER //

s

an r t n

mysql> CREATE PROCEDURE median (src_field VARCHAR (32), src_table VARCHAR (32)) -> SQL SECURITY INVOKER -> BEGIN -> SET @counter = 0; -> SET @stmt = CONCAT( -> 'SELECT FLOOR(AVG(table_a.', src_field,')) AS MEDIAN '> FROM (SELECT @counter := @counter+1 Count, ', -> src_field, -> ' FROM ', -> src_table, -> ' ORDER BY ', -> src_field, -> ') AS table_a, 127 '> (SELECT COUNT(*) totalrows FROM ',src_table, ') AS table_b '> WHERE Count IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1) '> '); -> PREPARE stmt_exec FROM @stmt; -> EXECUTE stmt_exec; -> DEALLOCATE PREPARE stmt_exec; -> END // Query OK, 0 rows affected (#.## sec)

e

bl a r fe

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

mysql> DELIMITER ; mysql> CALL median('Population','Continents'); +-----------+ | MEDIAN | +-----------+ | 482993000 | +-----------+ 1 row in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-13

MySQL Developer Techniques DELIMITER

128

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 5:Complex Calculations

Within the BEGIN … END syntax, statements must be terminated with a semicolon (;). This is due to the fact that this is the same default terminating character for SQL statements, it is important to change the SQL terminating character with the DELIMITER statement when using the MySQL command line client or batch processing. SQL SECURITY Stored routines bring a feature to database execution that is many times overlooked in discussions; this feature is the ability to give an end user a different privilege to execute a defined statement than they have in any other circumstance. This feature is implemented by use of the SQL SECURITY optional clause in the development of the stored routine: •

DEFINER – This value causes the routine to have the privileges of the user who created it (or another user set by the developer). This is the default if none is selected.

•

INVOKER – This value causes the routine to run with the privilege of the user who invoked it or is running the stored routine. This means the routine has access to database objects only if the invoker (user) already has access to them.

e

bl a r fe

s

an r t n

In the case of the median stored procedure, the SQL SECURITY needs to be set to INVOKER to allow the stored procedure to create the temporary table that is used and then to access that temporary table. If the SQL SECURITY setting is DEFINER (which is the default), then the temporary table would be created but then would not be able to be accessed by the remaining portions of the stored procedure.

g

no a s a h BEGIN … END ฺ n) idTheecompound ฺvtriggers. The BEGIN … END syntax can be used in stored routinesm and statements u (there is no limit) G within the BEGIN … END syntax can contain oneฺc oromore statements or no t 1 n statements (an empty compound statement is legal). 2 ts tude p s S SET @ s i n h t a value to defined variables using either = or := as the The SET statement allowsฺthe trauser tosassign e n assignment operator. o to u o m n ( ense Dynamic SQL a r T one of ltheicmost exciting features of MySQL stored routines is the use of server-side prepared t Perhaps e uy statements, which allows for an independent (from the API) method to create efficient and secure SQL

N Thi

for repeated execution. This in essence is a dynamic method to create SQL, thus the term Dynamic SQL. SQL syntax for prepared statements is based on three SQL statements: • PREPARE o An SQL statement is precompiled with PREPARE - The PREPARE statement prepares a statement and assigns it a name, stmt_name, by which to refer to the statement later. Statement names are not case sensitive. preparable_stmt is either a string literal or a user variable that contains the text of the statement. The text must represent a single SQL statement, not multiple statements. •

EXECUTE o Prepared statements are executed with the EXECUTE statement - After preparing a statement, it can only be executed with an EXECUTE statement that refers to the prepared statement name.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-14

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

•

Thi

Chapter 5:Complex Calculations

DEALLOCATE PREPARE o To deallocate a prepared statement, use the DEALLOCATE PREPARE statement An alternative to using DEALLOCATE PREPARE is DROP PREPARE. Attempting to execute a prepared statement after deallocating it results in an error. If you terminate a client session without deallocating a previously prepared statement, the server deallocates it automatically. {DEALLOCATE | DROP} PREPARE stmt_name

It is a good practice to deallocate resources in general if no longer needed. •

Stored functions are not able to utilize prepared statements, this is the reason that a stored procedure had to be used to obtain the median value.

CALL The CALL statement invokes a procedure that was defined previously with CREATE PROCEDURE. In the case of the median stored procedure, CALL median ('Population','Continents'), tells the median stored procedure to execute using the Continents tables Population column.

e

bl a r fe

ns a r t - is written, the User defined functions (UDF) contain code that becomes part of the running server, so when n a UDF o designer is bound by any and all constraints that otherwise apply to writing server code. UDF's written in C or atheynwillmustbebeinstalled C++ (or another language that can use C calling conventions), and the operating system on must s a h support dynamic loading. With that said, there are numerous repositories of UDF's that designers all over the world have ฺ n)haveisolved created and made available to the MySQL community. A number of designers the limitation of not having a e v ฺ d the requirements for a specific m doGnotu meet median aggregate function with their own UDF's. If the solutionso presented c database house, UDF's are another possible solution. 1ฺ ent 2 s pt Stud s @ thiThe s MySQL Training division offers a virtual training course n a r targeted to developers who would like to learn how ฺt use specifically n o to write MySQL user defined functions. This 8-hour eLearning o o t m course (broken up over 2 days), will provide MySQL users ( nse n with the skills necessary to write, manage and optimize a r ce T MySQL user defined functions. Additional information about i l t e this and any other training offering from the MySQL Training y u g division can be found at www.mysql.com/training. N MEDIAN UDF

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-15

MySQL Developer Techniques

5.6

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

129

Chapter 5:Complex Calculations

Simulating RANK When performing an analysis against a dataset, there are times when determining where a row of data fits in against the entire dataset. This is called ranking the data. However, There is no standard SQL solution that provides ranking. Other database vendors have created non-standard SQL functions to handle this limitation but as of yet, MySQL has not provided a function to solve this. With that said, using standard SQL there are a few ways to resolve this limitation and simulate the analytical ranking function in MySQL. Ranking without gaps The first way to simulate the ranking of rows is by creating a subquery that will be used to add an additional column to the dataset. This additional column would be responsible for determining the ranking by allowing more than one row to contain the same ranking without creating a gap in the ranking. For example, consider the following table: +--------+----------+-------+ | userid | username | score | +--------+----------+-------+ | 3 | Sarah | 97 | | 1 | Max | 95 | | 2 | Kai | 93 | | 5 | Pat | 93 | | 4 | Jeff | 72 | +--------+----------+-------+ In the case of this dataset, there are two users who have received the same score. Their rank (Kai and Pat) is a tie for 3rd, but does Jeff obtain 4th or 5th place in the ranking. Using the following standard SQL, Jeff would be assigned the ranking of 4th place (no gaps in the ranking):

e

bl a r fe

s

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den 130 s t sp s Stu @ i AS r_score, ( n score,thscore a mysql> SELECT userid, username, r t e ฺ -> SELECT COUNT(DISTINCT(score))+1 FROM scores us o>nr_score o -> WHERE score o t (m -> ) AS rank FROM scores se ORDER BY rank; n n a +--------+----------+-------+---------+------+ r e T| username ic | score | r_score | rank | | userid l t e y 3 | Sarah | 97 | u+--------+----------+-------+---------+------+ | 97 | 1 | g N | 1 | Max | 95 | 95 | 2 | i h

T

an r t n

| 2 | Kai | 93 | 93 | 3 | | 5 | Pat | 93 | 93 | 3 | | 4 | Jeff | 72 | 72 | 4 | +--------+----------+-------+---------+------+ 5 rows in set (#.## sec)

In this case, two additional columns are added to the table (r_score and rank). The r_score column is added to assist with the subquery that will determine the rank column by performing a "pseudo" self-join against the score column. The resulting ranking will eliminate any gaps in the ranks and provide identical ranking when ties (or more than one) occur in the column being ranked.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-16

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

131

Thi

Chapter 5:Complex Calculations

Ranking with gaps The second way to simulate the ranking of rows is similar to the first. There is still an additional column that is added to the dataset that will display the ranks, but rather than creating a subquery this solution utilizes WHERE clauses along with the ORDER BY clauses to manage the ranking. The big difference between the two solutions is that this solution will create a natural gap in the ranking and will assign any ties to a lower ranking in the scope of ranks. For example, consider the same table presented with the first solution: +--------+----------+-------+ | userid | username | score | +--------+----------+-------+ | 3 | Sarah | 97 | | 1 | Max | 95 | | 2 | Kai | 93 | | 5 | Pat | 93 | | 4 | Jeff | 72 | +--------+----------+-------+ In the case of this dataset, there is still the issue of two users who have received the same score. Jeff still would question if he obtains 4th or 5th place in the ranking. However, more importantly to the two individuals who tied (Kai and Pat), will they receive 3rd or 4th place. Using the following standard SQL, Jeff would be assigned the ranking of 5th place (gaps in the ranking):

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ mysql> SELECT t1.username, t1.score, COUNT(t2.score)+1 AS rank u< t2.score oONmt1.score G -> FROM scores AS t1 LEFT JOIN scores AS ฺt2 c t n DESC; -> GROUP BY t1.username, t1.score ORDER 21 BY dt1.score e s t +----------+-------+------+ sp s Stu | username | score | rank | @ +----------+-------+------+ n thi | Sarah | 97 | 1 ฺ|tra e | Max | 95 | on 2 | us o o | Kai | 93 | 3 | t m s3 e| | Pat | n (93 | n5 | a | Jeff | 72 | r e c T i l +----------+-------+------+ et 5 y rows in set (#.## sec) u g

N

The resulting ranking will include gaps in the ranks and provide identical ranking when ties (or more than one) occur in the column being ranked.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-17

MySQL Developer Techniques

5.7

Divide and Conquer Over the years, techniques have evolved that describe the process of breaking down a problem into manageable chunks. One such technique has come to be known as "Divide and Conquer". There is a formal mathematical model behind this technique, but basically it involves breaking down a complex problem into smaller, related steps, solving those smaller problems and then coalescing the results into the ultimate solution of the complex problem. The scenario Consider this scenario: In an effort to increase sales volume and customer loyalty, an online retailer of consumable products decided to offer a monthly purchase volume rebate to its customers who choose to register for this program. The rebate program would be a stepped program that works as follows:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

132

Thi

Chapter 5:Complex Calculations

133

•

Total volume of purchases calculated o Purchases accumulated over the period of a month are entitled to a variable, stepped rebate based upon the total "commvalue" volume purchased. This value is assigned to each product and the percentage of the total price of the product represented by this value varies from one product to another. This is done to make the rebate cost more "uniform" from the seller's point of view. What is "commvalue"?

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ • Rebate accumulated incrementally u G coismaccumulated ฺ t o The rebate is based on this volume1and incrementally. If a certain n 2 the customer e threshold level of purchases is tattained, is granted of a specified s pto that S tud amount. If a highera rebate percentage on all purchasessup threshold threshold amount of @ thatthmonth, purchases is attainednwithin is the volume above the lower threshold returns a a r higher percentage - otherwise the additional purchases above the original threshold level ฺtpercentage seof that n return the base level. If the lowest threshold level is not attained with u o o o t that months total purchases, no rebate is given. A sample rebate rate chart would be as m ( e n follows: ns a r e cCummulative T i l sales Rebate Achieved t e

Commvalue is short for commercial value. Calculating commvalue is done by removing the value of the product after the cost of packaging and other overhead, etc. along with excluding the amount the customer pays for shipping and handling. So products with simpler packaging have a higher percentage commvalue relative to the purchase price.

uy g N

250.00

5%

500.00

7%

1000.00

10%

1500.00

12%

2000.00

15%

So any additional purchases beyond the $2,000.00 level (which this retailer considers unlikely, but desirable) continue to earn at the highest rebate rate. A customer buying lots of products during a month gets an increasingly good deal, but never quite attains a rebate of 15% of the cv - and that is a substantially lower percentage of what was actually paid by the customer. Still, it is a generous offer for customer loyalty in a competitive market and can be an incentive to purchase more at the end of the month as well if a customer is tracking their cumulative purchases. _________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-18

MySQL Developer Techniques 134

Chapter 5:Complex Calculations

Calculating the rebate Here are some examples of how a rebate might be calculated:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

800.00 cumulative cv

Thi

= 250.00 * 5% + 250.00 * 7% + 300 * 7% = 12.50 + 17.50 + 21.00 = 51.00

1200.00 cumulative cv

e

bl a r fe

= 250.00 * 5% + 250.00 * 7% + 500.00 * 10% + 200.00 * 10% = 12.50 + 17.50 + 50.00 + 20.00 = 100.00

s

an r t n

o n a Notice that to even calculate these rebates, the purchase volume must be broken down into the amount s that falls into each percentage range to apply that percentage. This h isafollowed by summing those ฺ to perform the same intermediate values to obtain the total rebate amount. Any query created n) will ehave v ฺ d i feat. om t Gu c Step 1: Rebate percentage schedule table ฺ n 135 21schedule. e s Let's begin by characterizing the rebate percentage The simplest table that could hold these d t p andSpopulated tu as follows: sdefined threshold values and rebate rates might be @ s i n h a t r t mysql> CREATE TABLE commrate ( e s NULL, nฺ(8, 2)uNOT -> threshold DECIMAL o o o t -> rate DECIMAL (3,2) NOT NULL); (m nse n a r e could then be populated with the following values (250.00, .05), (500.00, .07), ctable T i l The commrate t uye(1000.00, .10), (1500.00, .12), (2000.00, .15): g N mysql> INSERT INTO commrate VALUES (250.00, .05), (500.00, .07), -> (1000.00, .10), (1500.00, .12), (2000.00, .15);

This would yield the following table: mysql> SELECT * FROM commrate; +-------------+------+ | threshold | rate | +-------------+------+ | 250.00 | 0.05 | | 500.00 | 0.07 | | 1000.00 | 0.10 | | 1500.00 | 0.12 | | 2000.00 | 0.15 | +-------------+------+

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-19

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

136

Chapter 5:Complex Calculations

But while this is descriptive, it does not offer enough tools for the task at hand. It would be more helpful if the endpoints were defined (or "bracketed") for each range, as well as the two rebate percentages that could potentially apply to each range. So a more thorough table definition for the rebate levels and rates that affords more flexibility might be: mysql> -> -> -> ->

CREATE TABLE commrate ( base DECIMAL(8,2) NOT NULL, threshold DECIMAL (8,2) NOT NULL, baserate DECIMAL (3,2) NOT NULL, rate DECIMAL (3,2) NOT NULL);

The *new* commrate table could then be populated with the following values (0, 250, 0, .05), (250, 500, .05, .07), (500, 1000.00, .07, .1), (1000, 1500, .1, .12), (1500, 2000, .12, .15): mysql> INSERT INTO commrate VALUES (0, 250, 0, .05), (250, 500, .05, .07), -> (500, 1000.00, .07, .1), (1000, 1500, .1, .12), -> (1500, 2000, .12, .15);

e

bl a r fe

ns a r t - month, But because someone might occasionally exceed $2,000.00 in cumulative purchases in n a given o there is also a need to have a row that defines that range. Safely assuming no one will buy $1,000,000.00 afor nthe data type of the cv in a single month, that upper bound could bet set as the maximum value s a threshold column: h ) n ideฺ v ฺ mysql> INSERT INTO commrate VALUES (2000, 999999.99,.15,.15); om t Gu c ฺ 21 den s t So the commrate table would now lookslike: p Stu @ is n h a t r mysql> SELECT * FROM commrate; ฺt use n +---------+-------------+----------+-----------+ o o | baserate | base | threshold to | bonusrate | m ( e +---------+-------------+----------+-----------+ n 250.00 ns| 0.00 | 0.05 | | 0.00 r|a e c T i500.00 | 0.05 | 0.07 | | 250.00 t || l1000.00 e y | 500.00 | 0.07 | 0.10 | u

g| 1000.00 |

N Thi

1500.00 | 0.10 | 0.12 | | 1500.00 | 2000.00 | 0.12 | 0.15 | | 2000.00 | 999999.99 | 0.15 | 0.15 | +---------+-------------+----------+-----------+

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-20

MySQL Developer Techniques

Step 2: Customer purchases table Now lets define the table that will hold the purchases made by customers. It is assumed that a customer could make more than one purchase in a given month, so there must be a column that holds the date of each purchase. Then there is a need to accumulate the total purchase amounts for a given month. For reference (and for other interesting statistical queries), there would need to be the full price of each purchase as well as the commvalue associated with it - and, of course, a pointer to the customer. The resulting table structure might be defined like this:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

137

Thi

Chapter 5:Complex Calculations

mysql> -> -> -> ->

CREATE TABLE purchases ( cuid CHAR (8) NOT NULL, saledate DATE NOT NULL, totalpurchase DECIMAL (8,2) NOT NULL, commvalue DECIMAL (8,2) NOT NULL);

There could be other columns in this table as well, such as an invoice number, but those other columns would not be involved in this calculation, so they will be left out for the sake of simplicity. The table could then be populated with some test data:

e

bl a r fe

s

mysql> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> ->

INSERT INTO purchases VALUES ('DFH34789', '2008-04-02', 82.27, 74.16), ('ADK89230', '2008-04-03', 36.82, 32.28), ('DFH34789', '2008-04-06', 187.36, 168.59), ('ADK89230', '2008-04-08', 95.48, 88.71), ('DFH34789', '2008-04-10', 83.90, 74.83), ('DFH34789', '2008-04-14', 61.62, 55.07), ('DFH34789', '2008-04-16', 57.75, 52.15), ('ADK89230', '2008-04-17', 114.65, 103.92), ('DFH34789', '2008-04-21', 85.36, 77.16), ('ADK89230', '2008-04-22', 41.25, 38.05), ('DFH34789', '2008-04-25', 118.83, 102.14), ('ADK89230', '2008-04-26', 98.20, 89.98), ('DFH34789', '2008-04-30', 132.66, 116.45), ('ADK89230', '2008-04-30', 62.69, 56.30), ('DFH34789', '2008-05-03', 97.78, 88.42), ('DFH34789', '2008-05-05', 236.41, 210.59), ('ADK89230', '2008-05-06', 45.44, 41.31), ('DFH34789', '2008-05-07', 37.92, 33.80), ('ADK89230', '2008-05-10', 82.35, 74.78), ('DFH34789', '2008-05-11', 61.72, 55.97), ('DFH34789', '2008-05-15', 77.15, 69.12), ('ADK89230', '2008-05-16', 51.64, 46.25), ('DFH34789', '2008-05-20', 375.86, 335.27), ('DFH34789', '2008-05-24', 68.16, 62.87), ('ADK89230', '2008-05-28', 120.18, 109.00), ('DFH34789', '2008-05-28', 92.30, 82.97), ('DFH34789', '2008-06-01', 112.57, 106.12), ('ADK89230', '2008-06-04', 68.79, 62.18), ('DFH34789', '2008-06-05', 217.62, 208.89), ('DFH34789', '2008-06-08', 86.32, 82.80), ('ADK89230', '2008-06-09', 85.22, 78.31), ('DFH34789', '2008-06-12', 64.72, 58.37), ('DFH34789', '2008-06-16', 47.18, 43.00), ('ADK89230', '2008-06-16', 122.37, 114.98), ('ADK89230', '2008-06-21', 91.38, 86.25), ('DFH34789', '2008-06-22', 225.86, 212.19), ('DFH34789', '2008-06-26', 98.15, 92.84), ('ADK89230', '2008-06-27', 148.00, 139.88), ('DFH34789', '2008-06-30', 102.40, 95.27);

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

an r t n

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-21

MySQL Developer Techniques 138

Chapter 5:Complex Calculations

Step 3: Calculate the rebate

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Now it is time to build a query that calculates the rebate for each customer for a given month. The first task is to build a query to accumulate the purchases for each customer for a given month. This is easy! A simple query that does this is:

Thi

mysql> SELECT cuid AS Customer, SUM(totalpurchase) AS MonthlyPurchases, -> SUM(commvalue) AS MonthlyCV FROM purchases -> WHERE saledate BETWEEN '2008-05-01' AND '2008-05-31' GROUP BY cuid; +----------+------------------+-----------+ | Customer | MonthlyPurchases | MonthlyCV | +----------+------------------+-----------+ | ADK89230 | 299.61 | 271.34 | | DFH34789 | 1047.30 | 939.01 | +----------+------------------+-----------+

e

bl a r fe

The column aliases are used here only to make the result set read nicely. Different "internal" aliases may be used as the statement is morphed to perform the rest of the task. The internal aliases may be needed to reference derived columns internally, but the ones shown here will be used again in the final result set. So all that is needed now is to add a column for "Rebate" and the task is finished. But that isn't as easy as it sounds! A number of intermediate tests will need to be performed before the final solution will be realized. The total purchase is not used in the rebate calculation, so that can be eliminated from the exploratory queries in this exercise and then can be added back in for the final result. To prepare to calculate the amount of each customer's cumulative purchases that falls into each rebate range, a cross join needs to be built between the result of the simple cumulative query and the commrate table, as follows:

s

an r t n

o

an s a h ) 139 n ideฺ v ฺ om t Gu c ฺ 1 n 2 e s d t mysql> SELECT * FROM ( spAS cvs Stu -> SELECT cuid, SUM(commvalue) @ nBY cuid thi -> FROM purchases GROUP a r t -> ) AS sls CROSS JOIN commrate ฺ seORDER BY cuid, base; n u o +----------+---------+---------+-----------+----------+------+ o | cuid | cv(m | base to | threshold | baserate | rate | se +----------+---------+---------+-----------+----------+------+ n n a r e | ADK89230 0.00 | 250.00 | 0.00 | 0.05 | T || 1162.18 lic || 250.00 t | ADK89230 1162.18 | 500.00 | 0.05 | 0.07 | e | y ADK89230 | 1162.18 | 500.00 | 1000.00 | 0.07 | 0.10 | u Ng| ADK89230 | 1162.18 | 1000.00 | 1500.00 | 0.10 | 0.12 | | ADK89230 | 1162.18 | 1500.00 | 2000.00 | 0.12 | 0.15 | | ADK89230 | 1162.18 | 2000.00 | 999999.99 | 0.15 | 0.15 | | DFH34789 | 2559.04 | 0.00 | 250.00 | 0.00 | 0.05 | | DFH34789 | 2559.04 | 250.00 | 500.00 | 0.05 | 0.07 | | DFH34789 | 2559.04 | 500.00 | 1000.00 | 0.07 | 0.10 | | DFH34789 | 2559.04 | 1000.00 | 1500.00 | 0.10 | 0.12 | | DFH34789 | 2559.04 | 1500.00 | 2000.00 | 0.12 | 0.15 | | DFH34789 | 2559.04 | 2000.00 | 999999.99 | 0.15 | 0.15 | +----------+---------+---------+-----------+----------+------+

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-22

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

140

Chapter 5:Complex Calculations

Then some additional columns can be injected to this result using a bit of logic to calculate the portion of the accumulated total for each customer that applies to each range as well as the rebate amount for that portion. Remember, the lower rebate percentage (baserate) applies to the portion if the next threshold level is not reached, but the higher percentage applies if that threshold is reached or exceeded: mysql> SELECT sls.*, crt.*, -> IF(cv ROUND(IF(cv (threshold - base) * rate)), 2) AS rpart -> FROM ( -> SELECT cuid, SUM(commvalue) AS cv FROM purchases pch GROUP BY cuid -> ) AS sls CROSS JOIN commrate AS crt ORDER BY cuid, base; +----------+---------+---------+-----------+----------+------+--------+-------+ | cuid | cv | base | threshold | baserate | rate | cvpart | rpart | +----------+---------+---------+-----------+----------+------+--------+-------+ | ADK89230 | 1162.18 | 0.00 | 250.00 | 0.00 | 0.05 | 250.00 | 12.50 | | ADK89230 | 1162.18 | 250.00 | 500.00 | 0.05 | 0.07 | 250.00 | 17.50 | | ADK89230 | 1162.18 | 500.00 | 1000.00 | 0.07 | 0.10 | 500.00 | 50.00 | | ADK89230 | 1162.18 | 1000.00 | 1500.00 | 0.10 | 0.12 | 162.18 | 16.22 | | ADK89230 | 1162.18 | 1500.00 | 2000.00 | 0.12 | 0.15 | 0.00 | 0.00 | | ADK89230 | 1162.18 | 2000.00 | 999999.99 | 0.15 | 0.15 | 0.00 | 0.00 | | DFH34789 | 2559.04 | 0.00 | 250.00 | 0.00 | 0.05 | 250.00 | 12.50 | | DFH34789 | 2559.04 | 250.00 | 500.00 | 0.05 | 0.07 | 250.00 | 17.50 | | DFH34789 | 2559.04 | 500.00 | 1000.00 | 0.07 | 0.10 | 500.00 | 50.00 | | DFH34789 | 2559.04 | 1000.00 | 1500.00 | 0.10 | 0.12 | 500.00 | 60.00 | | DFH34789 | 2559.04 | 1500.00 | 2000.00 | 0.12 | 0.15 | 500.00 | 75.00 | | DFH34789 | 2559.04 | 2000.00 | 999999.99 | 0.15 | 0.15 | 559.04 | 83.86 | +----------+---------+---------+-----------+----------+------+--------+-------+

e

bl a r fe

s

an r t n

T

no a s a h n) ideฺ v ฺ om t Gu c ฺ n 21base, dthreshold, e s t With the calculations being performed properly, the baserate, bonusrate and u pthe queryStotsimplify s cvpart columns can be removed from it in preparation for building the final 141 @ s i n h query that will yield only the r a eID (cuid) t number, cumulative cv and rebate (rpart): tcustomer's ฺ s n u SELECT sls.*, t m -> (cv - base) * baserate, se (threshold - base) * rate)), 2) AS rpart FROM ( n (cuid, n a -> SELECT SUM(commvalue) AS cv FROM purchases AS pch GROUP BY cuid r e ic JOIN commrate crt ORDER BY cuid, base; ->t )TAS sls lCROSS e y u+----------+---------+-------+ | cuid | cv | rpart | g N +----------+---------+-------+ i h | ADK89230 | 1162.18 | 12.50 | | ADK89230 | 1162.18 | 17.50 | | ADK89230 | 1162.18 | 50.00 | | ADK89230 | 1162.18 | 16.22 | | ADK89230 | 1162.18 | 0.00 | | ADK89230 | 1162.18 | 0.00 | | DFH34789 | 2559.04 | 12.50 | | DFH34789 | 2559.04 | 17.50 | | DFH34789 | 2559.04 | 50.00 | | DFH34789 | 2559.04 | 60.00 | | DFH34789 | 2559.04 | 75.00 | | DFH34789 | 2559.04 | 83.86 | +----------+---------+-------+

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-23

MySQL Developer Techniques Step 4: The Final Query

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

142

Thi

Chapter 5:Complex Calculations

Now the only task left is to sum the results from the various portions for each customer to yield the total rebate. The totpurch column will be added for the final display. The only problem is that an error will be produced if there is only a sum() function added around the rpart calculation and a "GROUP BY cuid" at the end. The reason for this is that the cv and totpurch columns are not involved in the grouping if only cuid is grouped. There are two choices available: •

Place a MAX() or MIN() or AVG() - although AVG() would then need to be rounded or truncated back to 2 decimal places to function around the cv and totpurch columns in the column list (which will also require additional aliasing for neatness).

• Group by both cv and totpurch in addition to cuid. The later will be chosen to complete the final query:

143

mysql> SELECT cuid AS Customer, totpurch AS MonthlyPurchases, cv AS -> ROUND(SUM(IF(cv (threshold - base) * rate))), 2) AS Rebate -> FROM ( -> SELECT cuid, SUM(totalpurchase) AS totpurch, SUM(commvalue) -> FROM purchases AS pch GROUP BY cuid -> ) AS sls CROSS JOIN commrate AS crt GROUP BY cuid, totpurch, +----------+------------------+-----------+--------+ | Customer | MonthlyPurchases | MonthlyCV | Rebate | +----------+------------------+-----------+--------+ | ADK89230 | 1264.46 | 1162.18 | 96.22 | | DFH34789 | 2811.87 | 2559.04 | 298.86 | +----------+------------------+-----------+--------+

MonthlyCV, baserate,

e

s

an r t n

AS cv

bl a r fe

ncv;o a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t p Stu table in this final statement rather than using Notice that each column was listed fromsthe sls derived @ s to each column. In addition, the query did not n sls.*. This is so that displaya aliases couldth beiassigned r ฺtcolumnsufrom qualify the names of derived se derived tables to keep their names distinct. n o o to m ( e an icens r T l et

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-24

MySQL Developer Techniques

Chapter 5:Complex Calculations

Further Practice Lab 5-A

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In a certain country employees pay 18.5% social charges, including pension, unemployment etc funds. On the remaining part they pay tax on revenue according to the following table:

Thi

Remaining Income

Tax Percentage

< 5,687

0.0%

5,687 - 11,344

5.5%

11,344 - 25,195

14.0%

25,195 - 67,546

30.0%

> 67,546

40.0%

Using the divide and conquer technique calculate the total tax, the tax on revenue % and the total % of the salaries that the employees get to keep when we have the following list of employees: Name

Salary

Johan

30,000

s

55,000

an r t n

o

n a s Tobias 90,000 a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e Max

e

bl a r fe

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-25

MySQL Developer Techniques

5.8

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

144

Thi

Chapter 5:Complex Calculations

Miscellaneous There are multiple miscellaneous SQL tricks that can come in handy when working with MySQL. The following list is just a few of them: Avoiding exact counts It is a common process for developers to use the SELECT COUNT(*) FROM tablename syntax to produce an exact count of rows. However, for most storage engines (especially InnoDB tables) this is costly in terms of performance. In many cases, the application does not require an exact count of rows, rather an approximate number would suffice. A less costly approach would be to utilize the INFORMATION_SCHEMA database with the following SQL:

mysql> SELECT table_rows FROM INFORMATION_SCHEMA.TABLES -> WHERE TABLE_NAME = 'table_name' AND TABLE_SCHEMA = 'table_schema';

145

Random rows Another common process that developers use is having the query statements choose random rows to return. A common approach, and one of the worst for performance, is to utilize the RAND() function within the query itself. The following is just one of many "poor" examples that are used by developers:

e

bl a r fe

s

an r t n

no a s with better performance. There are other ways to get a "random" row that produce the same outcomeabut For example, if the table has a numeric id column it is more efficient)toh calculate a random number and n ideฺ v then look up the row by the random value id. ฺ m Gu o c ฺ t real number of rows the mysql> SET @n = RAND() * 1000; -- substitute by n the 1 1000 2 e -> table likely has s pt Stud s @ tWHERE mysql> SELECT columns FROM table_name is id = @n; n h a r e which allows for overall better performance of indexes and ฺt usitself n This removes the function from the query o o tofeatures. other MySQL server optimization m ( e an icens r T l t e y gu mysql> SELECT columns FROM table_name ORDER BY RAND() LIMIT 1;

N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-26

MySQL Developer Techniques

5.9

Summary In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

146

Thi

Chapter 5:Complex Calculations

•

Simulate an aggregate multiplication function

•

Use variables to create a running total query

•

Utilize COALESCE to avoid divide by zero errors

•

Calculate the median value

•

Simulate the SQL RANK function in MySQL

•

Utilize SQL to solve complex problems

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

5-27

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 6

e

s

bl a r fe

an r t n

IMPROVING as a no h ฺ ) n PERFORMANCE ฺv uiOF de m o ฺc nt G 1 2 JOINS s de spt Stu

uy g N

@ this n a r ฺt use n o o to m ( e an icens r T l et

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

6

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

6.1

Thi

Chapter 6: Improving Performance of Joins

IMPROVING PERFORMANCE OF JOINS Overview This chapter provides the knowledge and skills necessary to improve the response time of queries that require the retrieval of data from more than one table. At the completion of this chapter, you will be able to:

148

•

Describe how queries are optimized by the MySQL server

•

Describe the different types of MySQL JOIN's

•

Describe how MySQL performs JOIN's

•

Explain how to improve the performance of JOIN's

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-1

MySQL Developer Techniques

6.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

149

Thi

Chapter 6: Improving Performance of Joins

Query Optimizer The task of the query optimizer in MySQL is to find the best or optimal plan for executing the SQL query that is submitted to the server. The query optimizer searches for the best plan among all the possible query evaluation plans. A process called greedy optimization enables the optimization engine to significantly reduce the amount of time it spends calculating optimal execution paths by a process of intelligent best-path reductions. For larger more complex queries, this can became a bottleneck in the server’s performance. The following example demonstrates the query process:

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o Query Optimizer (Techniques m se n n uses a list of known best practices to reduce not only the size of the query, but a As stated, the query optimizer r e c T i l more the time that the query takes to execute. The following is a small example of some of the etimportantly y many best practices that are implemented: u g

N

150

• Constant propagation - This best practice states that if the query is using constants (=, , =, , , LIKE) and there are transitive conjunction (x=y=z), then x must equal z (x=z). The optimizer will removes those inner equalities, thereby reducing the number of evaluations. For example:

mysql> SELECT Name FROM City WHERE Name = District AND District = 'San Jose';

... would be optimized to read ... mysql> SELECT Name FROM City WHERE Name = 'San Jose' AND District = 'San Jose';

This constant propagation best practice happens in a loop, so the output from one propagation step can be input for the next step. ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-2

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

• Dead code elimination - This best practice states that conditions that are always true or always false can be removed from the query execution. For example:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

mysql> SELECT Name FROM Country WHERE Name=Name AND Code='USA';

Thi

... would be optimized to read ... mysql> SELECT Name FROM Country WHERE Code='USA';

151

• Range queries - In queries that have a clear list of disjunctions, the optimizer will transform the query with the use of the IN clause. For example: mysql> SELECT Name FROM Country WHERE Code IN('USA','CAN','DEU');

... would be optimized to read ... mysql> SELECT Name FROM Country WHERE Code='CAN' OR Code='DEU' OR Code='USA';

e

bl a r fe

ns a r t Notice that the order of the where clause was also rearranged to have them sortednto- assist the optimizer. no a s more ranges meeting the • Range optimizer - In queries that have multiple ranges with oneaor h same criteria, the range that meets all the requirements will) be kept while n ideฺ the others will be v dropped. For example: ฺ om t Gu c ฺ mysql> SELECT Name FROM Country WHERE Population n AND Population > 2000; 21 d> e10000 s t sp s Stu ... would be optimized to read ... @ i n h a t r t e mysql> SELECT Name FROMnCountry s Population > 10000; o ฺ to uWHERE o (m nse n a r ce T i l t e guy

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-3

MySQL Developer Techniques

6.2.1

Greedy Optimizer The MySQL server contains a “greedy” optimizer which can greatly reduce the time required to arrive at a query execution plan for JOIN tables only. The “greedy” optimizer is most beneficial in cases where several tables are joined with no useful join key. In earlier versions of MySQL, the complexity of the search for a query execution plan was calculated as N!, where N is the number of tables to be joined. Below is an example of all the possible query execution plans for 4 tables without any useful join keys between them:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

152

Thi

Chapter 6: Improving Performance of Joins

Plan 1

Table1

Table2

Table3

Table4

Plan 2

Table1

Table2

Table4

Table3

Plan 3

Table1

Table3

Table2

Table4

Plan 4

Table1

Table3

Table4

Table2

Plan 5

Table1

Table4

Table2

Table3

Plan 6

Table1

Table4

Table3

Table2

Plan 7

Table2

Table1

Table3

Table4

Plan 8

Table2

Table1

Table4

Table3

Plan 9

Table2

Table3

Table1

Table4

Plan 10

Table2

Table3

Table4

Plan 22

Table4

Table2

Table3

Table1

Plan 23

Table4

Table3

Table1

Table2

Plan 24

Table4

Table3

Table2

Table1

e

bl a r fe

s

an r t n

no Table1 a s Table3 Table2 Table4 Table1 Plan 11 a h Table2 Table4 Table3 Plan 12 n) ideฺTable1 v ฺ Table3 Table1 Plan 13 mTable2Gu Table4 o c ฺ t Table3 Table11 Table4 Table1 Plan 14 n 2 e s d t Table3 Table2 Table1 Table4 Plan 15 u p t s S Table3 @ Table2 Table4 Table1 Plan 16 is n h a t Table3 Table4 Table1 Table2 Plan 17 r t e ฺ s n u Table4 Table2 Table1 Plan 18 oo Table3 o t m Table4 Table1 Table2 Table3 Plan(19 e s n n a Table1 Table3 Table2 r Plan 20lice Table4 T t Table4 Table2 Table1 Table3 Plan 21 e

uy g N

A more intense example of the complexity can be seen with a join involving 20 tables: 153

N! or 20! = 2,432,902,008,176,640,000 The “greedy” optimizer reduces this to N!/(N-D)!, where D is the maximum depth of the search. In other words, the depth determines how far into the “future” the optimizer should look in order to evaluate whether the query execution plan should be expanded further. In the event of a depth that is set to 7, the following complexity will be realized: N!/(N-D)! => 20!/(20-7)! => 20!/13! = 390,700,800 Thus reducing the complexity by a factor approximately 6x10^9.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-4

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Reducing query execution plans is not a guarantee

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Although the greedy optimizer does not guarantee the best possible of all query execution plans (this is currently being worked on), it can reduce the time spent arriving at a query execution plan for a join involving a great many tables — 30, 40, or more — by a factor of as much as 1,000. This should eliminate most if not all situations where users thought that the optimizer had hung when trying to perform joins across many tables.

Thi

optimizer_prune_level

154

The optimizer_prune_level variable tells the optimizer to skip certain plans based on estimates of the number of rows accessed for each table. This kind of “educated guess” rarely misses optimal plans, and may dramatically reduce query plan generation times. That is why this option is on (optimizer_prune_level=1) by default. However, this option can be switched off (optimizer_prune_level=0) with the risk that query plan generation may take much longer. Note that, even with the use of this heuristic, the optimizer still explores a roughly exponential number of plans. optimizer_search_depth

e

bl a r fe

s

an r t n

The optimizer_search_depth variable tells how far into the “future” of each incomplete plan the optimizer should look in order to evaluate whether it should be expanded further. Smaller values of optimizer_search_depth may result in orders of magnitude smaller query plan generation times. For example, queries with 12, 13, or more tables may easily require hours and even days to generate if optimizer_search_depth is close to the number of tables in the query. At the same time, if generated with optimizer_search_depth is set to 3 or 4, the query plan may generate in less than a minute for the same query. If you are unsure of what a reasonable value is for optimizer_search_depth, this variable can be set to 0 to tell the optimizer to determine the value automatically. The default value set is 62.

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-5

MySQL Developer Techniques

6.3 155

Chapter 6: Improving Performance of Joins

EXPLAIN The EXPLAIN command describes how the MySQL optimizer has decided to execute a SELECT statement and access the data. The information that is displayed can be used in several ways:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

• EXPLAIN can provide information that points out the need to add an index.

Thi

• If a table already has indexes, EXPLAIN can be used to find out whether the optimizer is using them. • If indexes exist but aren't being used, query rewrites in different ways can be tested prior to executing them. EXPLAIN will provide information that can be used to determine if the rewrites will perform better by helping the server use the available indexes. • When performing a JOIN, EXPLAIN will provide what type of JOIN is being performed and provide additional details that can provide assistance in improving the performance of the JOIN. When using EXPLAIN to analyze a query, it's helpful to have a good understanding of the tables involved. DESCRIBE can be used to obtain information about a table's columns, and SHOW INDEX can be used to display information about its indexes.

e

bl a r fe

s

an r t n

EXPLAIN works with SELECT queries, but can be used in an indirect way for UPDATE and DELETE statements as well: A SELECT statement can be written that has the same WHERE clause as the UPDATE or DELETE and use EXPLAIN to analyze the SELECT.

no a s a h Using EXPLAIN n) query ethatฺ is to be analyzed For v To use EXPLAIN, place the keyword EXPLAIN in front of the ฺSELECT d 156 i u command: omwith thet G example, the following SELECT statement will be evaluated EXPLAIN c ฺ 21 den WHERE Language='English'\G s t mysql> EXPLAIN SELECT CountryCode FROM CountryLanguage p Stu *************************** 1. row s *************************** @ is id: 1 n h a t r select_type: SIMPLE ฺt use n table: CountryLanguage o o to type: index m ( e possible_keys: ns an NULL r key: PRIMARY e c T tkey_len: 33li e y ref: NULL u g rows: 984 N Extra: Using where; Using index 1 row in set (#.## sec)

157

The output of this explain tells the end user: • select_type - the query execution is considered SIMPLE (it does not contain subqueries or unions). • table - the query execution is using only one table (CountryLanguage). • type - that the server will complete a full table scan against the index (instead of against the data rows themselves). • possible_keys - that there are no possible (NULL) candidate keys to utilize to satisfy the query.

• key - the PRIMARY key will be used to satisfy this query (in spite of the fact that the optimizer did not list this as a possible candidate key) ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-6

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

• key_len - the length (measured in bytes) of the key that will be used. In this case the length of the primary key is CountryCode is 3 bytes and the length of the Language column is 30 bytes.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

• ref - that neither a constant nor another column is being used (NULL), indicating selection by an expression or range of values.

Thi

• rows - the optimizer's approximation of how many rows from the table (or index in this case) it will need to examine when the actual query execution takes place. • Extra - the query optimizer will use a WHERE clause to identify rows that satisfy the query (Using where) and can optimize the query by reading values from the index without having to read the corresponding data rows (Using index). 158

Extending EXPLAIN Adding the word EXTENDED after the EXPLAIN command will provide not only additional information to the EXPLAIN output, but will also provide additional information into the SHOW WARNINGS output. mysql> EXPLAIN EXTENDED SELECT CountryCode FROM CountryLanguage -> WHERE Language='English'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: CountryLanguage type: index possible_keys: NULL key: PRIMARY key_len: 33 ref: NULL rows: 984 filtered: 100.00 Extra: Using where; Using index 1 row in set, 1 warning (#.## sec)

e

bl a r fe

s

an r t n

N

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e nฺ us o o o t mysql> SHOW WARNINGS\G (m nse 1. row *************************** *************************** n a Level: Note r ce T i l t Code: 1003 ye select `world`.`countrylanguage`.`CountryCode` AS `CountryCode` from guMessage: `world`.`countrylanguage` where (`world`.`countrylanguage`.`Language` = _latin1'English') 1 row in set (#.## sec)

The filtered column indicates an estimated percentage of table rows that will be filtered by the table condition. In this case, the query optimizer will need to check every row of the index and thus no table (or index) rows will be filtered. Issuing the SHOW WARNINGS command after running the EXPLAIN EXTENDED command will produce information on how the optimizer qualifies tables and column names in the SELECT statement, what the SELECT looks like after the application of rewriting and optimization rules, and possibly other notes about the optimization process.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-7

MySQL Developer Techniques

6.4

JOIN's In the world of databases and SQL, there are three types of JOIN's that are the most prevalent: • Inner join - A join that identifies combinations of matching rows from two tables is called an inner join. The joining is done by connecting one table, using the primary key, and another table that references it with a foreign key.

mysql> SELECT L_Name, F_Name, Class_Name, Class_Date -> FROM Students INNER JOIN Classes ON Students.Student_ID = Classes.Student_ID -> WHERE Student_ID = '100459';

... or ... mysql> SELECT L_Name, F_Name, Class_Name, Class_Date FROM Students, Classes -> WHERE Students.Student_ID = Classes.Student_ID AND -> Student_ID = '100459';

e

bl a r fe

s

an r t n

no a se a at h D ) nnt_)IDlass_ideฺ v ฺ ,C u om(StudtemeG c ฺ 21 dessn_Na s t sp s Stu Cla @ i n h a t r t e nฺ | MySQL Allievo | Sakila 2008-06-09 us for Database Administrators || 2008-06-23 o o Allievo | Sakila t|oMySQL Performance Tuning (m| Sakila Allievo | 2008-07-12 se | MySQL High Availability n n a r e lic et T D) e _I m nt Na de _ tu , F ( S me Na L_

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

159

Thi

Chapter 6: Improving Performance of Joins

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-8

MySQL Developer Techniques

• Cross join – This type of join is where each row of each table is crossed with each row in every other table to produce all possible combinations. The result is known as a ‘Cartesian Product’. A Cartesian product is a mathematical (set theory) concept which basically means combining all elements from the first entity with all elements of the second entity. In the example below, each student will show up for each class in the classes table thus creating a roster where every student is "assigned" to every possible class in the university:

Da te

mysql> SELECT L_Name, F_Name, Class_Name, Class_Date -> FROM Students CROSS JOIN Classes;

L_ (Stu Na de me nt_ , F ID) _N am e

Cla ss_ (Stude Na me nt_ID) ,C las s_

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

160

Thi

Chapter 6: Improving Performance of Joins

e

bl a r fe

s

an r t n

o

an s a h ) n| 2008-06-09 eฺ v Allievo | Sakila | MySQL for Datab ... ฺ d i m... |G2008-06-09 u Dude | Pete | MySQL for Datab o c ฺ t Hangover | Dee | MySQL for ... n | 2008-06-09 2for1 Datab e s Smith | Joe | MySQL Datab ... | 2008-06-09 d t p tu ... | 2008-06-09 Allievo | Sakila | s MySQL forSDatab @ s ... an | eMySQL thifor Trees | 2008-12-15 r t Dude | ฺPete on| Deeto u|sMySQL for Trees | 2008-12-15 Hangover o Smith (m ns| eJoe | MySQL for Trees | 2008-12-15 n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-9

MySQL Developer Techniques

• Outer join - Whereas an INNER JOIN will find combinations of matching rows from joined tables, the OUTER JOIN also finds the instances where a row in one table has no match in another table. For example, the following SQL using a left outer join would look for students who have a Student_ID but have not attended any classes:

L_ (Stu Na de me nt_ , F ID) _N am e

(S Co tuden l um t_ ns ID) Ou tpu te d

mysql> SELECT L_Name, F_Name -> FROM Students LEFT JOIN Classes ON Students.Student_ID = Classes.Student_ID -> WHERE Class_Name IS NULL;

e

bl a r fe

s

an r t n

Da te

No

no a s a h n| )Dee ideฺ v ฺ Hangover m Gu o c ฺ twould look for classes that have no For example, the following SQL using a 1 right outer join n 2 e 162 s students enrolled: pt Stud s @ this mysql> SELECT Class_Name, Class_Date n a r -> FROM Students RIGHT e ON Students.Student_ID = Classes.Student_ID ฺt JOIN Classes -> WHERE L_Name o ISnNULL; us o to m ( e an icens r T l t e y gu N hi

T

No (S Co tud lum ent ns _ID Ou ) tpu ted

Cla ss_ (Stude Na me nt_ID) ,C las s_

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

161

Chapter 6: Improving Performance of Joins

MySQL for Dogs – A real howler | 2008-11-12 MySQL and Trees – From leaf to bud | 2008-12-15

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-10

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Inline Lab 6-A In this lab you will use the country table in the world database to execute a left and right JOIN.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Step 1: Examine the columns that make up the country table

Thi

Open a new terminal window and start the MySQL client. After you have successfully logged into the MySQL client, utilize the world database by entering the following SQL statement: mysql> USE world;

In the MySQL client, type the following MySQL DESC statement to list the columns in the country table: mysql> DESC Country\G +----------------+-------------------------------------| Field | Type +----------------+-------------------------------------| Name | char(52) | Continent | enum('Asia','Europe','North Ameri ... | Region | char(26) | SurfaceArea | float(10,2) | IndepYear | smallint(6) | Population | int(11) | LifeExpectancy | float(3,1) | GNP | float(10,2) | GNPOld | float(10,2) | LocalName | char(45) | GovernmentForm | char(45) | HeadOfState | char(60) | Capital | int(11) | Code2 | char(2) +----------------+-------------------------------------16 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t eThe DESC command produces a listing of the columns contained with the country table to include the

uy field name and the field type (other information is produced but not displayed here). g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-11

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 2: Examine the columns that make up the city table

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In the same terminal window from step 1, enter the following SQL statement to list the columns in the city table:

Thi

mysql> DESC City; +-------------+----------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+----------+------+-----+---------+----------------+ | ID | int(11) | NO | PRI | NULL | auto_increment | | Name | char(35) | NO | | | | | CountryCode | char(3) | NO | | | | | District | char(20) | NO | | | | | Population | int(11) | NO | | 0 | | +-------------+----------+------+-----+---------+----------------+ 5 rows in set (#.## sec)

e

bl a r fe

s

The DESC command produces a listing of the columns contained with the city table. All of the fields listed from the DESC command are displayed here.

an r t n

no a s a h In the same terminal window from step 2, enter the following SQL statement to list all the countries and n) using eanฺINNER JOIN: their corresponding capitals that are located in the 'Eastern Africa' region v ฺ d i m Gu o c ฺ mysql> SELECT Country.Name AS Country, City.Name nt= City.ID 21 dASeCapital -> FROM Country INNER JOIN City ON tCountry.Capital s -> WHERE Region = 'Eastern Africa'; sp s Stu +------------+--------------+ @ i n h a | Country | Capital | t r t e +------------+--------------+ onฺ t|o us | Burundi | Bujumbura o | Djibouti | Djibouti (m e || | Eritrea an | Asmarans r | Addis ceAbeba | | Ethiopia T i l t e | Kenya | Nairobi | | gu| yComoros | Moroni Step 3: Obtain values from both the city and the country tables using an outer join

N

| Madagascar | Antananarivo | | Malawi | Lilongwe | ... | Tanzania | Dodoma | | Uganda | Kampala | | Zimbabwe | Harare | +------------+--------------+ 19 rows in set (#.## sec)

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-12

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Enter the following SQL statement to list only the countries and their corresponding capitals that are located in the 'Eastern Africa' region and have a capital assigned using a LEFT JOIN:

Thi

mysql> SELECT Country.Name AS Country, City.Name AS Capital -> FROM Country LEFT JOIN City ON Capital = ID -> WHERE Region = 'Eastern Africa'; +--------------------------------+--------------+ | Country | Capital | +--------------------------------+--------------+ | Burundi | Bujumbura | | Djibouti | Djibouti | | Eritrea | Asmara | | Ethiopia | Addis Abeba | | Kenya | Nairobi | | Comoros | Moroni | | Madagascar | Antananarivo | | Malawi | Lilongwe | ... | Tanzania | Dodoma | | Uganda | Kampala | | Zimbabwe | Harare | | British Indian Ocean Territory | NULL | +--------------------------------+--------------+ 20 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-13

MySQL Developer Techniques

6.4.1

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

163

Thi

Chapter 6: Improving Performance of Joins

Nested JOIN's MySQL uses what is known as a Nested Loop Join Algorithm and is the simplest join-algorithm to implement. This join-algorithm compares the outer data set (usually a table) row by row with inner data set rows that match the join-condition. The diagram below represents a visual sampling of what takes place in the join of the country and city tables:

mysql> SELECT Country.Name, City.Name AS Capital FROM Country JOIN City -> ON Country.Capital=City.ID; world.Country +----------------------+---------+ | Afghanistan | 1 | | Netherlands | 5 | | Netherlands Antilles | 33 | | Albania | 34 | | Algeria | 35 | | American Samoa | 54 | | Andorra | 55 | | Angola | 56 | | Anguilla | 62 | | Antigua and Barbuda | 63 | | United Arab Emirates | 65 | | Argentina | 69 | | Armenia | 126 | | Aruba | 129 | | Australia | 135 | | Azerbaijan | 144 | | Bahamas | 148 | | Bahrain | 149 | | Bangladesh | 150 | ... +----------------------+---------+

world.City +----+------------------+ | 1 | Kabul | | 2 | Qandahar | | 3 | Herat | | 4 | Mazar-e-Sharif | | 5 | Amsterdam | | 6 | Rotterdam | | 7 | Haag | | 8 | Utrecht | | 9 | Eindhoven | | 10 | Tilburg | | 11 | Groningen | | 12 | Breda | | 13 | Apeldoorn | | 14 | Nijmegen | | 15 | Enschede | | 16 | Haarlem | | 17 | Almere | | 18 | Arnhem | | 19 | Zaanstad | | 20 | ┤s-Hertogenbosch | | 21 | Amersfoort | | 22 | Maastricht | | 23 | Dordrecht | | 24 | Leiden | | 25 | Haarlemmermeer | | 26 | Zoetermeer | | 27 | Emmen | | 28 | Zwolle | | 29 | Ede | | 30 | Delft | | 31 | Heerlen | | 32 | Alkmaar | | 33 | Willemstad | ... +----+------------------+

e

bl a r fe

s

an r t n

N

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t ethis representation, the world.country table is the outer data set and the world.city table is the In guyinner data set. For every record in the country table, the city table is scanned to determine if there is a

match based on the join criteria. From this diagram, it is easy to see that this type of join-algorithm can have performance issues that must be addressed. MySQL implements a 'single-sweep multi-join' algorithm which is a variation of the nested loop join algorithm. In addition, the remainder of this chapter will focus on how to work with this algorithm and the MySQL server to improve the performance of joins.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-14

MySQL Developer Techniques

6.4.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

164

Thi

Chapter 6: Improving Performance of Joins

Self Joins One of the more interesting joins is those joins that utilize the same table for both the outer query and the inner query thus creating a self-join. The most common reason to use a self-join is for evaluating relationships between different columns of data in the same table. For example, the following SQL uses a self-join to locate all the countries that gained independence in the same year as Qatar:

mysql> SELECT x1.IndepYear, x1.Name, x2.Name -> FROM Country AS x1 JOIN Country AS x2 -> ON x1.IndepYear = x2.IndepYear AND x1.Name='Qatar'; +-----------+-------+----------------------+ | IndepYear | Name | Name | +-----------+-------+----------------------+ | 1971 | Qatar | United Arab Emirates | | 1971 | Qatar | Bahrain | | 1971 | Qatar | Bangladesh | | 1971 | Qatar | Qatar | +-----------+-------+----------------------+ 4 rows in set (#.## sec)

e

s

N

1971 1971 1971 1971

| | | |

Qatar Qatar Qatar Qatar

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e world.Country +-----------+----------------------+ | IndepYear | Name | +-----------+----------------------+ | 1919 | Afghanistan | | 1581 | Netherlands | | 1912 | Albania | | 1962 | Algeria | | 1278 | Andorra | | 1975 | Angola | | 1981 | Antigua and Barbuda | | 1971 | United Arab Emirates | ... | 1991 | Russian Federation | | 1945 | Vietnam | | 1991 | Estonia | | 1776 | United States | | 1980 | Zimbabwe | +-----------+----------------------+ 192 rows in set (#.## sec)

guy

bl a r fe

world.Country +-----------+----------------------+ | IndepYear | Name | +-----------+----------------------+ | 1919 | Afghanistan | | 1581 | Netherlands | | 1912 | Albania | | 1962 | Algeria | | 1278 | Andorra | | 1975 | Angola | | 1981 | Antigua and Barbuda | | 1971 | United Arab Emirates | ... | 1991 | Russian Federation | | 1945 | Vietnam | | 1991 | Estonia | | 1776 | United States | | 1980 | Zimbabwe | +-----------+----------------------+ 192 rows in set (#.## sec)

| | | |

United Arab Emirates Bahrain Bangladesh Qatar

The most important aspect to realize from this SQL, is that aliases must be setup (... AS ...) to distinguish the outer and inner tables from each other. This type of join is very common in databases where hierarchical data is being stored and retrieved.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-15

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

165

FULL OUTER JOIN

In addition to the left outer join and right outer join, there is a join that combines the results of both left and right outer joins called a full outer join. In a full outer join, the resulting joined table will contain all records from both tables, and fill in NULLs where there are holes (missing matches) on either side.

L_ Na me ,F

Sakila Pete Dude Joe NULL NULL NULL NULL

| | | | | | | |

Aviello Dude Hangover Smith NULL NULL NULL NULL

_N am e

| | | | | | | |

NULL NULL NULL NULL MySQL MySQL MySQL MySQL

Cla ss_ Na me ,C las s_D ate

6.4.3

Chapter 6: Improving Performance of Joins

| | | | for Database Adm ... | High Availability | for Dogs | for Trees |

NULL NULL NULL NULL 2008-06-09 2008-07-12 2008-11-12 2008-12-15

e

bl a r fe

s

an r t n

no a s a h n) fromidboth In this example a FULL OUTER JOIN, which displays all the records eฺtables without any real v ฺ connecting join column, is represented. om t Gu c ฺ n MySQL does not support FULL OUTER JOIN's21 e s d t tu sp s S 166 MySQL does not support the use of a @ FULL OUTER JOIN (or FULL JOIN) directly; however, there are n to correctththisi limitation. The most common workaround is to create workarounds that can be implemented a r t e query which allows the combination of result sets from two ฺ a UNION an SQL statement that utilizes sALL n u o o or more SELECT queries: to m ( e s n an* FROM r e mysql> SELECT Students_noid AS x LEFT JOIN Classes_noid AS y c T li ->t ON x.F_Name = y.Class_Name e uy -> UNION ALL g -> SELECT * FROM Students_noid AS x RIGHT JOIN Classes_noid AS y hi N -> ON x.F_Name = y.Class_Name;

T

This SQL statement would handle any duplicates correctly without producing any invalid outputs. It is necessary to use UNION ALL. If UNION alone was used, duplicates would be eliminated.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-16

MySQL Developer Techniques

6.5

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

167

Thi

Chapter 6: Improving Performance of Joins

Improving JOIN performance A JOIN operation, in any database system, is a complex process that requires database developers, designers and administrators to work together to ensure that the data, design and system all support the most optimal environment for executing these operations. With that said, before discussing improving the performance of JOIN's, the following question needs to be asked: "Is a JOIN operation absolutely necessary?" Normalization Normalization of data in a database is concerned with minimizing the amount of data that is duplicated to safeguard the database from data incongruity. Ensuring a database is most effectively normalized for the data that it contains is an important task that database designers must consider. However, those responsible for designing databases do not usually take into account the performance of the database when the actual data needs to be retrieved by joining multiple tables.

e

bl When a database is over-normalized, there may be a need to denormalize the data down to a level that willera sf improve the performance of mission critical queries. In addition, there is a growing number of database n a developers that are using denormalization to simplify complex queries and to reduce the table -trcount and n row accesses. So the question you have to ask yourself is this: "Is optimizing this JOINo the best approach or should I consider denormalizing my data to a level where a JOIN is not necessary a n or using minimal s tables?" Denormalization is especially useful in many web applications orhwhen a using the Cluster storage ) engine (NDB). n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e guy Sometimes the best way to optimize a JOIN is by not doing a JOIN at all!

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-17

MySQL Developer Techniques

6.5.1

Reducing Rows Touched by Joins The query optimizer will attempt to reduce the number of rows that need to be retrieved in JOIN operations. The following demonstration shows how this works:

168

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 6: Improving Performance of Joins

mysql> EXPLAIN SELECT Country.Name, City.Name FROM Country JOIN City -> ON Country.Code = City.CountryCode\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4079 Extra: *************************** 2. row *************************** id: 1 select_type: SIMPLE table: Country type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 3 ref: world.City.CountryCode rows: 1 Extra: 2 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ n switchedththei order of the tables (Country and City) to reduce the In this example, the query optimizer a r t e outcome is 4079 rows will be touched in the first table while ฺbe touched. number of rows that would The s n u o only one row will need o to be touched to in the second table, thus 4079 x 1 equals only 4079 rows having to be touched. (m e an icens r T l t e y u

g

N Thi

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-18

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

169

Thi

Chapter 6: Improving Performance of Joins

Forcing a specific table order Forcing the SELECT statement to keep the order entered by using the STRAIGHT_JOIN syntax produces a different result: mysql> EXPLAIN SELECT Country.Name, City.Name FROM Country STRAIGHT_JOIN City -> ON Country.Code = City.CountryCode\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: Country type: ALL possible_keys: PRIMARY key: NULL key_len: NULL ref: NULL rows: 239 Extra: *************************** 2. row *************************** id: 1 select_type: SIMPLE table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4079 Extra: Using where 2 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ Using the STRAIGHT_JOIN syntax thatithe left table is read before the right table. This can be n ensures h a t r used for those (few) cases for which the join optimizer puts the tables in the wrong order. If the query t e ฺ n tables, thisusexample would have been the outcome. The first table would have optimizer did not reorderothe o ttheo second table would have resulted in touching 4079 rows. This would required touching 239 rows and m ( e s being touched all together. It is clear from this example that the reordering of resultedainn239 x 4079nrows r e c of theT tables by the query optimizer was in the best interest of performance. li t e guy

N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-19

MySQL Developer Techniques

6.5.2

Covered Searches One of the dangers in performance is asking for more than is needed. For example, a common approach seen in executing SELECT statements is to request a complete listing of all columns in the table using the SELECT * syntax. This can produce poor query execution times and produce a list of other problems (such as external applications expecting a specific column in a specific position in the output). Even if the output includes indexed columns, those non-indexed columns require the query to search the data pages for the output resulting in a performance degradation. The best way to ensure performance of JOIN's and non-JOIN's is to only use indexed columns both in the output and the WHERE clauses. In this way, the query only needs to access the indexed pages and is referred to as a "covered search". The following example demonstrates how this would affect what is sent to the query optimizer to execute:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

170

Thi

Chapter 6: Improving Performance of Joins

171

mysql> EXPLAIN EXTENDED SELECT Country.*, City.* FROM Country JOIN -> City ON Country.Code = City.CountryCode AND -> GovernmentForm='Republic'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4079 filtered: 100.00 Extra: 172 *************************** 2. row *************************** id: 1 select_type: SIMPLE table: Country type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 3 ref: world.City.CountryCode rows: 1 filtered: 100.00 Extra: Using where 2 rows in set, 1 warning (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-20

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

173 mysql> SHOW WARNINGS\G *************************** 1. row *************************** Level: Note Code: 1003 Message: select `world`.`country`.`Code` AS `Code`, `world`.`country`.`Name` AS `Name`,`world`.`country`.`Continent` AS `Continent`,`world`.`country`.`Region` AS `Region`, `world`.`country`.`SurfaceArea` AS `SurfaceArea`, `world`.`country`.`IndepYear` AS `IndepYear`, `world`.`country`.`Population` AS `Population`, `world`.`country`.`LifeExpectancy` AS `LifeExpectancy`, `world`.`country`.`GNP` AS `GNP`,`world`.`country`.`GNPOld` AS `GNPOld`,`world`.`country`.`LocalName` AS `LocalName`, `world`.`country`.`GovernmentForm` AS `GovernmentForm`, `world`.`country`.`HeadOfState` AS `HeadOfState`, `world`.`country`.`Capital` AS `Capital`,`world`.`country`.`Code2` AS `Code2`,`world`.`city`.`ID` AS `ID`,`world`.`city`.`Name` AS `Name`,`world`.`city`.`CountryCode` AS `CountryCode`, `world`.`city`.`District` AS `District`,`world`.`city`.`Population` AS `Population` from `world`.`country` join `world`.`city` where ((`world`.`country`.`GovernmentForm` = _latin1'Republic') and (`world`.`country`.`Code` = `world`.`city`.`CountryCode`)) 1 row in set (#.## sec)

e

bl a r fe

ns a r t Not only does the SELECT statement become expanded to handle each column in both the tables, - but the n o WHERE statement now must search the table data versus being able to execute the query only using the n a performance a indexed data. With larger tables with more data and possible more columns, this can create s a drain. The better solution is to choose only columns that are actually needed in the output and to keep the h WHERE clauses to indexed fields only. n) ideฺ v ฺ Use of GROUP BY and ORDER BY om t Gu 174 c ฺ nof the query execution. Furthermore, 21performance e As stated, accessing the indexes only can improve s d t p BYSexpressions tu limiting indexes to the GROUP BY and/orsORDER can also improve the performance of @ queries. Even if it is unrealistic tonlimit the output of the query to indexed columns, limiting the s i columns onlyonly h a t r GROUP BY and/or ORDER ฺBY expressions to indexed can still have a positive impact on t e s n performance. o to u o m n ( ense a r T lic t e uy g hi N

T

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-21

MySQL Developer Techniques

6.5.3

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

175

Thi

Chapter 6: Improving Performance of Joins

Double Checking EXPLAIN The MySQL server maintains many status variables that provide information about its operation. For the most part, database administrators have been the end users most interested in these variables; however, as a developer, many of these variables can provide great insight into improving (and verifying any improvement) the execution of queries. In the case of using the EXPLAIN command, server variables can be used to double check that the EXPLAIN operation is producing the correct query execution plan. The following is a list of server status variables that can be used in performing this double checking procedure: • Handler_read_first - This server variable displays the number of times the first entry was read from an index. If this value is high, it suggests that the server is doing a lot of full index scans. • Handler_read_key - This server variable displays the number of requests to read a row based on a key. If this value is high, it is a good indication that the tables used in query executions are properly indexed.

176

e

bl • Handler_read_next - This server variable displays the number of requests to read the next row in ra fe key order. This value is incremented if the query is using an index column with a range constraint or s n if performing an index scan. tra n othe previous row • Handler_read_prev - This server variable displays the number of requests to read n a in key order. This read method is mainly used to optimize ORDER BY ... s DESC. a h • Handler_read_rnd - This server variable displays the number of)requests toฺ read a row based on a fixed position. This value is high if the server is executing queries ฺvn thaturequire ide sorting of the result. A m high number here would identify that the server is c executing a lotG of queries that require entire table o 1ฺ ent scans or there are joins that don't use keys properly. 2 s t displays udthe number of requests to read the next row • Handler_read_rnd_next - This server pvariable t s S in the data file. This value will be lot of table scans. Generally this suggests that the @high ifththere is areindexed n a tables in the query executions are not properly or that the queries being executed are not r tof the indexes. e ฺ s written to take advantage n o to u o m • Select_full_join This e server variable displays the number of joins that perform table scans because ( use indexes. s n they do not This value should be 0. If this value is not 0, table indexes should be n a r e c T i checked and used. l et

uy • g N

Select_full_range_join - This server variable displays the number of joins that used a range search on a reference table.

• Select_range - This server variable displays the number of joins that used ranges on the first table. This is normally not a critical issue even if the value is quite large. • Select_range_check - This server variable displays the number of joins without keys that check for key usage after each row. This value should be 0. If this value is not 0, table indexes should be checked and used. • Select_scan - This server variable displays the number of joins that did a full scan of the first table.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-22

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Inline Lab 6-B

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this lab you will verify that EXPLAIN command produces a correct evaluation of the query execution by looking at a query's effect on SHOW SESSION STATUS.

Thi

Step 1: Review the indexes of the City table In the MySQL client, type the following MySQL SHOW INDEXES statement to list the indexes in the city table (located in the world database): mysql> SHOW INDEX FROM city\G *************************** 1. row *************************** Table: city Non_unique: 0 Key_name: PRIMARY Seq_in_index: 1 Column_name: ID Collation: A Cardinality: 4079 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ m Gu o c ฺ t This index is the primary key and The city table contains one index which is applied to 1the ID column. n e contains 4079 values in the set (Cardinality). ts2 p Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-23

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 2: Determine how the query optimizer will execute a join using the City and Country tables

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In the same terminal window from step 1, enter the following SQL statement to list the columns in the city table: mysql> EXPLAIN SELECT SQL_NO_CACHE country.Code, country.Name, -> city.Name FROM country JOIN city ON country.Code = -> city.CountryCode WHERE city.CountryCode LIKE 'A%'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: city type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4079 Extra: Using where *************************** 2. row *************************** id: 1 select_type: SIMPLE table: country type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 3 ref: world.city.CountryCode rows: 1 Extra: 2 rows in set (#.## sec)

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e nฺdeterminesusthat the best way to execute this query is to swap the order of the The EXPLAIN command o o tables and perform a table scanto on the city table and a single row lookup on the country table for each m ( e s n matching city row. The SQL_NO_CACHE clause is added for verification purposes to ensure that we npicture of what the server a an accurate r e c are T seeing is doing. li t e uy

g

N Thi

e

bl a r fe

Step 3: Verify the EXPLAIN output by performing the steps of the join individually

In the same terminal window from step 2, enter the following SQL statement to count how many records are contained in the city table: mysql> SELECT COUNT(*) FROM city; +----------+ | COUNT(*) | +----------+ | 4079 | +----------+ 1 row in set (#.## sec)

This output verifies that the EXPLAIN got it right in reference to how many rows would have to be evaluated in the city table. ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-24

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 4: Review the WHERE clause of the earlier JOIN

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In the same terminal window from step 3, enter the following SQL statement to determine how many matches would fit the criteria of the earlier WHERE clause:

Thi

mysql> SELECT COUNT(Code) FROM country WHERE Code LIKE 'A%'; +-------------+ | COUNT(Code) | +-------------+ | 17 | +-------------+ 1 row in set (#.## sec)

Since "city.CountryCode LIKE 'A%'" filters country codes during the table scan, the server should only need to touch the Country table when the server knows there is a match. Additionally, if the city table matches a record in the country table already retrieved in the result set, the server does not need to retrieve the record twice. This can be seen in the next few steps, but before we evaluate the SHOW SESSION STATUS, we must first flush the logs with the following command:

e

bl a r fe

s

an r t n

no a s a h n) ideฺ Step 5: Execute and evaluate the SELECT statement v ฺ u (which will subsequently omSQLt statements In the same terminal window from step 4, enter the following G c ฺ add fill the status variables): 21 den s t tu sp scity.Name S mysql> SELECT country.Code, country.Name, @ i nON country.Code -> FROM country JOIN city = city.CountryCode h a t r t -> WHERE city.CountryCode LIKE 'A%'; e ฺ s +------+----------------------+--------------------------------+ oon to|uName | Code | Name (m | e s +------+----------------------+--------------------------------+ n a icen r | AFG |TAfghanistan | Kabul | l ... et yAUT | Austria | Klagenfurt | gu|+------+----------------------+--------------------------------+ mysql> FLUSH STATUS; Query OK, 0 rows affected (#.## sec)

N

107 rows in set (#.## sec)

mysql> SHOW SESSION STATUS LIKE 'Handler_read%'; +-----------------------+-------+ | Variable_name | Value | +-----------------------+-------+ | Handler_read_first | 0 | | Handler_read_key | 15 | ... | Handler_read_rnd_next | 4080 | +-----------------------+-------+ 6 rows in set (#.## sec)

The values, though slightly off, show that the EXPLAIN provided the correct "guess" as to how the SELECT statement would perform against the server. ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-25

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 6: Determine why session values not completely accurate

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

The first value that we see slightly off is the Handler_read_key variable. In step 4, we determined that the value should read 17; however, in our output in step 5 it reads 15. In the same terminal window from step 5, enter the following SQL statement to see why this might be the case: mysql> SELECT DISTINCT country.Code FROM country -> WHERE Code NOT IN (SELECT CountryCode FROM city); +------+ | Code | +------+ | ATA | | ATF | | BVT | | HMD | | IOT | | SGS | | UMI | +------+ 7 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ m serverGissued As far as the Handler_read_rnd_next variable, theo mysql u a final operation at the end c ฺ t of the table scan to detect when no more rows were available. This n 4079).of course caused an additional 21of 4080d(versus e operation to be recorded and thus gave us the tvalue s sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t uye

This SELECT statement shows that there are some countries without cities, including two starting with A which our query did not need to touch. So when these two country codes are removed, the Handler_read_key variable is displaying an accurate number.

g

N Thi

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-26

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Step 7: Prevent table scan

Thi

The query being executed in this lab was performing a complete table scan which can be disadvantegous when it comes to performance. This table scan can be eliminated by adding an index to the CountryCode column in the city table. In the same terminal window used in step 6, enter the following SQL statements: mysql> ALTER TABLE city ADD INDEX (CountryCode); Query OK, 4079 rows affected (0.38 sec) Records: 4079 Duplicates: 0 Warnings: 0 mysql> EXPLAIN SELECT SQL_NO_CACHE country.Code, country.Name, city.Name -> FROM country JOIN city ON country.Code = city.CountryCode WHERE -> city.CountryCode LIKE 'A%'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: city type: range possible_keys: CountryCode key: CountryCode key_len: 3 ref: NULL rows: 104 Extra: Using where

e

bl a r fe

s

an r t n

o

N

an s a h ) n ideฺ v ฺ om t Gu *************************** 2. row *************************** c ฺ id: 1 21 den s select_type: SIMPLE t sp s Stu table: country @ i type: eq_ref n h a t r possible_keys: PRIMARY t e nฺ key: PRIMARY us o o o key_len: 3 t (m nse ref: world.city.CountryCode n a rows: r 1 e T Extra: lic t e gu2 yrows in set (#.## sec)

With the city.CountryCode being indexed, the resulting query will have to only search through this index; thus eliminating a full table scan.

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-27

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 8: Execute and evaluate the SELECT statement again

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

With the city.CountryCode being indexed, lets execute the SELECT statement against the server to determine if the EXPLAIN was correct in its evaluation. In the same terminal window used in step 7, enter the following SQL statements:

Thi

mysql> SELECT COUNT(*) FROM city WHERE CountryCode LIKE 'A%'; +----------+ | COUNT(*) | +----------+ | 107 | +----------+ 1 row in set (#.## sec)

Our EXPLAIN was slightly off but still in a very close percentage to be considered a very good "guess" on the number of rows that would need to be touched by the query execution. Before we can evaluate the SHOW SESSION STATUS, we must again flush the logs with the following command: mysql> FLUSH STATUS; Query OK, 0 rows affected (#.## sec)

e

bl a r fe

s

an r t n

N

no a It is now time to actually execute the query that we performed an EXPLAIN s on to verify that the a h EXPLAIN was providing an educated "guess" on the results: n) ideฺ v ฺ mysql> SELECT SQL_NO_CACHE country.Code, country.Name, u WHEREFROM om tcity.Name G c -> country JOIN city ON country.Code = city.CountryCode ฺ -> city.CountryCode LIKE 'A%'; 21 den s t +------+----------------------+--------------------------------+ sp s Stu | Code | Name |@ Name | i n +------+----------------------+--------------------------------+ h t tra | Oranjestad e | ABW | Aruba | ฺ s n o to u ... o | AZE | Azerbaijan | (m nse | MingΣτevir +------+----------------------+--------------------------------+ n a r set l(#.## 107 rowsTin ce sec) i t ye SHOW SESSION STATUS LIKE 'Handler_read%'; gumysql> +-----------------------+-------+ | Variable_name | Value | +-----------------------+-------+ | Handler_read_first | 0 | | Handler_read_key | 16 | | Handler_read_next | 107 | ... | Handler_read_rnd_next | 0 | +-----------------------+-------+ 6 rows in set (#.## sec)

The Handler_read_next variable (which verify's that an index dataset was used) provides an accurate count to what was expected and the Handler_read_key is again only slightly over due to the servers need to verify that the end of the file was reached. In any case, the values are either right on or close enough to considered an excellent "guess" on the part of the EXPLAIN command. ______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-28

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 9: Other status variables to consider

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

While the Handler% status variables are quite handy, there is another set of server variables that are also quite useful. In the same terminal window used in step 8, enter the following SQL statement:

Thi

mysql> SHOW SESSION STATUS LIKE 'Select%'; +------------------------+-------+ | Variable_name | Value | +------------------------+-------+ | Select_full_join | 0 | | Select_full_range_join | 0 | | Select_range | 1 | | Select_range_check | 0 | | Select_scan | 0 | +------------------------+-------+ 5 rows in set (#.## sec)

The Select_range server variable has been incremented to show that the last query execution used a range on the first table.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-29

MySQL Developer Techniques

6.5.4

JOIN - Best Practices In addition to the many tools that have been discussed in relation to JOIN's in this chapter, there are other best practices that should be taken into account when considering joining two or more tables:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

177

Thi

Chapter 6: Improving Performance of Joins

• Column types - When performing JOIN operations, the smaller the indexed columns are in size the better performance will be realized. In addition, numeric column types are considered to be better JOIN candidates than string column types when considering performance. • Take advantage of memory - If possible, it is best to fit the data sets that are being accessed in the join into memory. Processing in memory is much faster than accessing hardware and will pay out large dividends in performance improvement. • Temporary tables - Storing portions of data that is being accessed in temporary tables can limit the amount of data needing to be accessed and thus improve the performance of JOIN's utilizing that data.

e

bl a r fe

• Full table scans are not necessarily bad - When it comes to working with large data sets, full table scans are often faster than range scans.

s

an r t n

• Inner Join preferred - Do not use an outer join if an inner join can be used. An outer join forces the order, while an inner join allows the optimizer to choose.

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-30

MySQL Developer Techniques

6.5.5

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

178

Chapter 6: Improving Performance of Joins

Simulating INTERSECT and MINUS Users of other database systems may be familiar with the INTERSECT and MINUS (sometimes implemented as EXCEPT) commands. These commands are not available in MySQL; however, the logic behind these two commands can easily be simulated in MySQL. INTERSECT An INTERSECT is simply an inner join where the data of one table is compared with those of the another table, and those that match are displayed with any duplicates being weeded out. A typical implementation of INTERSECT would look something like this:

mysql> SELECT Name, Region FROM country_a -> INTERSECT -> SELECT Name, Region FROM country_b;

e

bl a r fe

Even though MySQL does not specifically have the INTERSECT command, the following SQL statement will produce the same result:

s

an r t n

T

no a s a h n) ideฺ MINUS v ฺ m taking u the results from the first SQL 179 ostatements The outcome of a MINUS operation is to compare two SQL G c ฺ t 1second SQLenstatement. If the second SQL statement statement and removing any duplicates found in the 2 s t includes values not found in the first SQL statement, they uared simply passed over and not displayed. A p t s S typical implementation of MINUS would @look something is like this: n h a t r ฺt FROMucountry_a se n mysql> SELECT Name, Region o o -> MINUS to country_b; m ( e -> SELECT Name, Region FROM an icens r T l does not specifically have the MINUS command, the following SQL statement will t though MySQL e Even y gu produce the same result: N hi mysql> SELECT DISTINCT country_a.Name, country_a.Region -> FROM country_a INNER JOIN country_b -> USING (Name, Region);

mysql> SELECT DISTINCT country_a.Name, country_a.Region -> FROM country_a LEFT JOIN country_b -> USING (Name, Region) WHERE country_b.Name IS NULL;

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-31

MySQL Developer Techniques

6.6

Summary In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

180

Thi

Chapter 6: Improving Performance of Joins

•

Describe how queries are optimized by the MySQL server

•

Describe the different types of MySQL JOIN's

•

Describe how MySQL performs JOIN's

•

Explain how to improve the performance of JOIN's

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

______________________________________________________________________________________________ ______________________________________________________________________________________________ ______________________________________________________________________________________________

6-32

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 7

e

s

bl a r fe

an r t n

no HIERARCHICAL a s a h n) ideฺ v DATA ฺ u om

uy g N

ฺc nt G 1 2 e pts Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

Chapter 7: Hierarchical Data

7. HIERARCHICAL DATA 7.1. Overview This chapter provides the knowledge and skills necessary to understand and create tables that utilize tree and hierarchical structures along with the ability to write queries to retrieve the data. At the completion of this chapter, you will be able to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

182

Thi

•

Define graphs, trees and hierarchies

•

Create and modify an adjacency list structure

•

Create and modify nested set structures

•

Create and modify path enumeration structures

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-1

MySQL Developer Techniques

Chapter 7: Hierarchical Data

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

7.2. Graphs, Trees and Hierarchies

Thi

183

Storing and representing data can take on many forms. One of the more interesting approaches to storing data is through the use of nested sets. Nested sets are the answer to the branch of mathematics that deal with graph theory, or abstract structures. The theory behind the mathematics is not as important as the implementation and thus only a brief introduction of the concept will be presented. With that said, there are three main graph types: •

Graphs - The term "graph", when being used to describe abstract data structures, consists of nodes (sometimes referred to as vertices) and relationships defining how the nodes are related to each other (sometimes referred to as connections). The following example represents what a graph data structure can look like:

1

2

3

4

5

6

e

7

8

9

10

11

12

bl a r fe

s

an r t n

no a 13 14 15 16 17 as18 h ฺ ) n ฺv uide m o G 24 t 20 19 21 1ฺc 23 22 n 2 e pts Stud s @ thatthareisconnected to each other through paths in the node This graph contains 24 n nodes a r structure. If the starting take a minimum of 5 steps to reach the 24 node ฺt the node sisevia9, itthewould n (traversing through graph following nodes: 10, 16, 22, 23, 24). This graph u o o to however, if it was a data structure for a maze it would come across pathway may look strange; m ( e ns aasnlogical: r e c T li et th

184

uy g N

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-2

MySQL Developer Techniques •

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

185

Thi

Chapter 7: Hierarchical Data

Trees - A tree data structure is a specialized graph that works on the principal that there is a starting point (referred to as root node or parent node) that contains sub-ordinate nodes (referred to as child nodes). The root node contains data that can be further broken down into more detailed data in the child node. Child nodes can also have child nodes that can break down its "parent" node data even further. The following example demonstrates how a tree data structure could be represented:

Earth

North America

South America

Europe

Asia

Africa

Australia

e

bl a r fe

s

Canada

Central America

Island Nations

no a ...s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t u being the root node) would be a child From this example, the countries p t(Earth s ofIf thethesworld S node to the habitable continents. countries are smaller in size, they may be grouped @ i North America). n h together (such as Island Nations tunder In addition, there can also be a r tof the data e ฺ s further breakdown to include the states/territories each country (such as n o to uStates). This type of tree data structureof would Alabama o under the United make it easier to m ( e find the state Alabama by digging down through the Earth North America - United States s one node (or data table) containing all the data. n enhaving a branch versus r lic about trees: et T Generalizations Alabama

186

an r t n

United States

uy g N

Alaska

Arizona

Arkansas California

Trees contain only one path between any two nodes. Thus Canada (a country located in North America) can not be connected to both North America and the United States (another country located in North America) at the same time. o Every node must be connected to another node in the tree structure. In trees there are no disconnected nodes. o When a node is deleted, something must be done to connect any "orphaned" nodes to the tree or all the sub-ordinate nodes will become trees themselves. For example, if the North America node was deleted, Canada would become the root node for its own tree along with Central America, Island Nations and the United States nodes all becoming root nodes and independent trees themselves no longer being connected to the Earth root node. o Each node is unique to the other nodes. Thus the same node can not be connected to more than one parent node. With that said, a node that would have the same name such as the country "Georgia" and the state "Georgia" should be uniquely identified in some means to avoid any confusion. _________________________________________________________________________________________________ o

_________________________________________________________________________________________________ _________________________________________________________________________________________________

7-3

MySQL Developer Techniques •

187

Hierarchical - A hierarchical data structure is a specialized graph structure that contains root and child nodes but have two advantages over trees: o

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 7: Hierarchical Data

o

Inheritance - When a node is removed, sub-ordinate nodes automatically are promoted to the deleted nodes level. For example, in the case of the leadership of the United States, if something happened to the President, the Vice President would be promoted to President, thus inheriting the position. Multiple-role nodes - In a general tree structure, each node is unique. In a hierarchical structure, an individual node can take on one or more roles. This is common in organization charts (where one individual can have many different roles in the organization) or a parts breakdown (where one part can be used over again). The following example demonstrates a parts breakdown for a basic birdhouse:

e

bl a r fe

s

an r t n

u

g N i h

T

no a as Apparel Electronics Health Sports Fitness Toys Electronics Apparel Health&&Beauty Beauty Sports)&&h Fitness ฺ Toys n e v ฺ m GuidFitness o Portable c Baby Cosmetics 1ฺ ent Equipment Music 2 s d t sp s Stu @ i Hair n TV & Portable h a t Female r t Video Music Care e ฺ s n u o o to m ( e Video Team anMaleicens Games Pharmacy r Sports T l yet

Building Blocks

Board Games

Video Games

In this fictious super store, there are 5 different departments (Apparel, Electronics, Health & Beauty, Sports & Fitness and Toys). Under each department there are different groups of products that each department is responsible for. In the case of inheritance, if the apparel department grew in size, each of the groups could become a department of themselves (Apparel would be removed and the three groups underneath apparel would be promoted to departments themselves). In the case of multiple-role nodes, the portable music group underneath the electronics department is also present underneath the sports & fitness department (along with video games being located under both electronics and toys).

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-4

MySQL Developer Techniques

Chapter 7: Hierarchical Data

7.3. Adjacency List Structures 188

For databases (or any computer program) to recognize and manage graphs, the system must be able to identify how the individual nodes relate to each other. There are multiple ways to accomplish this. The first way is to use what is known as an adjacency list.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Undirected graphs

Thi

Undirected graphs are those sets of nodes that either have straight lines signifying that there is no directional flow or those graphs that have arrows on both ends of the connector showing that the flow can be either way. The following table would represent an adjacency list for an undirected graph with five nodes: Node

Connection

A

B,C

B

A,D

C

A,E

D

B,E

E

C,D

B A

D C

E

e

bl a r fe

s

an r t n

no a Creating a database table to resemble the above adjacency list, requires a tablesthat is denormalized: 189 a h ) n ideฺ mysql> CREATE TABLE ud_graph (Node CHAR(1), Connect ฺCHAR(1)); v u om t G('B','A'), c ฺ mysql> INSERT INTO ud_graph VALUES ('A','B'), ('A','C'), 1 n 2('D','B'), e -> ('B','D'), ('C','A'), ('C','E'), ('D','E'), s d t -> ('E','C'), ('E','D'); sp s Stu @ i n h a t mysql> SELECT Node, Connect FROM ud_graph; r t e +------+---------+ onฺ to us | Node | Connect | o (m nse +------+---------+ | A | Bran | e i | A t |TC |c l | y Be | A | u| B | D | g N |C |A | | C | E | | D | B | | D | E | | E | C | | E | D | +------+---------+ 10 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-5

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

190

Thi

Chapter 7: Hierarchical Data

Directed graph The directed graph resembles the undirected graphs with the exception that the flow is from one node to any other node in a one-way directional pattern (the node the connection goes to will not return via the same path). This adjacency list requires a little more detail: Node

Enter

Exit

A

B

D

B

E

C

B

E

A

C

B

F

D

A

E

E

D

B

E

F

B

A

B

C

D

E

F

e

C

E

s

an r t n

no a srequires a table a Creating a database table to resemble the above adjacency list, also 191 h denormalized: n) ideฺ v ฺ u om G c mysql> CREATE TABLE d_graph (Node CHAR(1), Enter CHAR(1), Exit CHAR(1)); ฺ t 21 den s t mysql> INSERT INTO d_graph VALUES ('A','B','D'), tu ('B','E','C'), sp('D','A','F'), S -> ('B','E','A'), ('C','B','F'), ('E','D','B'), @ s i n h -> ('E','F','B'), ('F','C','E'); a t r ฺt use n o mysql> SELECT * FROM o d_graph;to m ( +------+-------+------+ se n | Exit n | Node | Enter | a r e +------+-------+------+ lic | tT | y Ae | B | D u Ng|| BB || EE || CA || F

bl a r fe

that is

| C | B | F | | D | A | F | | E | D | B | | E | F | B | | F | C | E | +------+-------+------+ 7 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-6

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

192

Thi

Chapter 7: Hierarchical Data

Trees

A

Trees resemble the directed graph but with the direction flowing down from the root node. With trees appears the idea of parent nodes. To create an adjacency list for a tree, the idea of a parent node must be incorporated: Node

B

C

D

F

G

H

Parent

A B

A

C

A

D

A

E

B

F

B

G

C

E

e

bl a r fe

s

an r t n

no a s a In this case, the root node (the node on the top of the tree) is A and hashno parent node. This will be taken into account when creating the table in MySQL by allowingn into the eaฺNULL value v and) placing dnormalized 193 i parent field associated with the root node. In addition, the m treeฺtable canube by having the G node column as a primary key with no null's allowed. ฺco t 1 n 2 s tuKEYdeNOT NULL, Parent CHAR(1)); ptPRIMARY mysql> CREATE TABLE tree (Node CHAR(1) s S @ s i n h t ('B','A'), ('C','A'), mysql> INSERT INTO tree VALUES tra ('A',NULL), e ฺ s n -> ('D','A'), ('E','B'), ('F','B'), ('G','C'), ('H','C'); o to u o m se mysql> SELECTn*(FROM tree; n a +------+--------+ r e c T i l | Nodet | Parent | ye +------+--------+ u g N | A | NULL | H

C

| B | A | | C | A | | D | A | | E | B | | F | B | | G | C | | H | C | +------+--------+ 8 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-7

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

194

Thi

Chapter 7: Hierarchical Data

Hierarchy

A

Hierarchy data sets resemble trees but with the two additional features; inheritance and multiple-role nodes. The following adjacency list demonstrates how a node (such as a part used more than once in a design) can be used in more than one branch: Node

B

C

D

F

E

G

Parent E

A B

A

C

A

D

A

E

B

F

B

E

C

e

bl a r fe

s

an r t n

no a s limitation of hierarchy a Now of course, this hierarchy is not in normalized form, but that is an inherent h ฺ row is NULL. Note models. Node A is the root node for this hierarchy, thus the parent this 195 n)field for econtains v ฺ d i that node E is used in two different branches of the hierarchy and thus the table two rows with m Gu o the same node ID (similar to the graph data sets discussed earlier). c 1ฺ ent 2 s pt SParent tud CHAR(1)); mysql> CREATE TABLE hierarchy (NodesCHAR(1), @ this n a mysql> INSERT INTO hierarchy VALUES ('A',NULL), ('B','A'), ('C','A'), r t ('F','B'), e ฺ s n -> ('D','A'), ('E','B'), ('E','C'), ('G','C'); o to u o m se mysql> SELECTn*(FROM hierarchy; n a +------+--------+ r e c T i l | Nodet | Parent | ye +------+--------+ u g N | A | NULL | G

C

| B | A | | C | A | | D | A | | E | B | | F | B | | E | C | | G | C | +------+--------+ 8 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-8

MySQL Developer Techniques

7.3.1.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

196

Thi

Chapter 7: Hierarchical Data

Search Tree Depths

Once the hierarchical data has been stored in the database, it is time to develop a means to query the data. There are multiple activities that may be necessary for the end administrator to accomplish concerning the data, with the most likely being focused on producing a full tree of the data. Displaying the complete hierarchy Using the hierarchy table produced in the last section, it is possible to use a self-join query against the table to produce an indented display of the entire tree:

A

E

mysql> SELECT t1.Node AS level1, t2.Node AS level2, t3.Node AS level3 -> FROM hierarchy AS t1 -> LEFT JOIN hierarchy AS t2 ON t2.Parent = t1.Node -> LEFT JOIN hierarchy AS t3 ON t3.Parent = t2.Node; +--------+--------+--------+ | level1 | level2 | level3 | +--------+--------+--------+ | A | B | E | | A | B | F | | A | C | E | | A | C | G | | A | D | NULL | | B | E | NULL | | B | F | NULL | | C | E | NULL | | C | G | NULL | | D | NULL | NULL | | E | NULL | NULL | | F | NULL | NULL | | E | NULL | NULL | | G | NULL | NULL | +--------+--------+--------+ 14 rows in set (#.## sec)

B

C

D

F

E

G

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ceit is possible to see each branch of the hierarchy. The following describe what the T i l t From this query, e

uy g N

rows are showing in detail: •

First through fourth rows - The first row, shows a branch of the tree which consists of starting with the root node (A), traversing down through node B into the last node of the branch (E). The last node of the branch (containing no child nodes) is called a leaf node. The next row follows this same pattern but this time traversing the A, B and F nodes. The third and fourth rows traverse the A and C branch to get to the E and G leaf nodes.

•

Fifth through ninth rows - These rows provide some interesting information about the branches but with a twist. Level2 of these rows display all the leaf nodes for the hierarchy, thus node D (which has no children) is associated with the leaf nodes located in the third level of the hierarchy. So in essence, these rows show the branches that contain a leaf node and their parent node: node D is the leaf node connected to node A, Node E is the leaf node for both node B and C, node F is the leaf node connected to node B and node G is the leaf node connected to node C.

Remaining rows (tenth through fourteenth) - These rows display all the leaf nodes for the hierarchy in the first column. _________________________________________________________________________________________________ •

_________________________________________________________________________________________________ _________________________________________________________________________________________________

7-9

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

197

Chapter 7: Hierarchical Data

Leaf nodes only There are times when it is beneficial to only view the leaf nodes of the hierarchy. This can be accomplished through the following self-join query: mysql> SELECT DISTINCT t1.Node -> FROM hierarchy AS t1 -> LEFT JOIN hierarchy AS t2 -> ON t1.Node = t2.Parent -> WHERE t2.Parent IS NULL; +------+ | Node | +------+ | D | | E | | F | | G | +------+ 4 rows in set (#.## sec)

A

E

B

C

D

F

E

G

e

bl a r fe

ns a r t Similar to the output of the last five rows in the previous query (4 rows are due to n the- SELECT o DISTINCT option that eliminated duplication), this query displays only those nodes n that are leaf a nodes (they have no child nodes connected to them). s a h 198 Another way to accomplish this is by using a subquery to obtain) all the parent and then n idisenotฺ alsonodes comparing those parent nodes against the child nodes. If there ฺisva node that a parent node, than it is a leaf node and will be displayed: om t Gu c ฺ 21 den s t mysql> SELECT DISTINCT Node FROM Hierarchy sp s Stu -> WHERE Node NOT IN @ i n FROM Hierarchy -> (SELECT DISTINCT Parent WHERE Parent IS NOT NULL); h a t r t e +------+ ฺ on to us | Node | o +------+ (m nse | D | n a | E |Tr lice | F et | yG | u|+------+ g hi N 4 rows in set (#.## sec)

T

Subqueries are less performance friendly, but the code and logic in creating them can be advantageous when creating more complex queries to mine the data.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-10

MySQL Developer Techniques

Chapter 7: Hierarchical Data

Lab 7-A

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this hands on lab, you will query the adjacency list hierarchical data of the world database.

Thi

Lab Preparation - The contents of the world database will be used to demonstrate the capabilities of using hierarchical data throughout this chapter. However, the data of the world database is not in a hierarchical format. The following steps will need to be accomplished to create hierarchical tables based on the data in the world database: 1. Create a new database called hierarchy: mysql> CREATE DATABASE Hierarchy;

2. In the Hierarchy database just created, source the location_hierarchy.sql file (Note: Your instructor will identify the correct location to find this file, for this example the /tmp directory is used):

e

bl a r fe

s

an r t n

no a seach type of hierarchical a This will create (and populate with data) the following tables. Note, h structure associated with the tables will be discussed in detail n)throughout eฺthe remainder of this v ฺ d i chapter: m Gu couses ฺ • LocationAdjacencyList - This1table thet tree approach to hierarchies with n 2 e each node being a child node containing a reference its parent node (except the root s da uniqueto identifier u pthet table includes t node itself). The content of called LocationID, s S @ s i the reference to thenparent called ParentLocationID and two supporting columns h a t r t called LocationName and LocationType. e onฺ to us o • LocationMaterializedPath - This table uses the path enumeration approach to m ( e hierarchies which stores the path from the root node to the individuals node being sThis approach is possible because n referenced. n a every node will have a unique path to the r e lic tT

mysql> SOURCE /tmp/location_hierarchy.sql;

e guy

N

root and thus the path becomes a unique identifier. This table also has the same unique identifier called LocationID along with the two supporting columns called LocationName and LocationType; however, instead of a parent column referencing the unique indentifier a string column called LocationPath stores the path of each node from the root node. This column will be used to identify where each node fits in the hierarchy structure.

• LocationNestedSet - This table uses the nested set hierarchical structure which stores the left and right edges for each individual node with the root node containing the furthest most left edge (usually identified by the numeric value of 1) and the furthest most right edge. This table also has the same unique identifier called LocationID along with the two supporting columns called LocationName and LocationType. However, with this table there is a left (Lnode) and right (Rnode) column which identifies the edges which will be used to parse the hierarchy.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-11

MySQL Developer Techniques 1.

Querying the adjacency list hierarchical data of the world database Determining the root node in an adjacency list hierarchical data can be accomplished by querying the data for the row that does not contain a parent identification:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

1.

Thi

Chapter 7: Hierarchical Data

mysql> SELECT * FROM LocationAdjacencyList -> WHERE ParentLocationID IS NULL; +------------+------------------+--------------+--------------+ | LocationID | ParentLocationID | LocationName | LocationType | +------------+------------------+--------------+--------------+ | 1 | NULL | Earth | Planet | +------------+------------------+--------------+--------------+ 1 row in set (#.## sec)

The output identifies that the root node for this table is the planet Earth with a location identifier of 1 and no parent connection. 2. Locate all the children nodes that are directly connected to the root node with the following SQL statement: mysql> SELECT children.* FROM LocationAdjacencyList root -> INNER JOIN LocationAdjacencyList children -> ON root.LocationID = children.ParentLocationID -> WHERE root.ParentLocationID IS NULL; +------------+------------------+---------------+--------------+ | LocationID | ParentLocationID | LocationName | LocationType | +------------+------------------+---------------+--------------+ | 5 | 1 | Africa | Continent | | 7 | 1 | Antarctica | Continent | | 2 | 1 | Asia | Continent | | 3 | 1 | Europe | Continent | | 4 | 1 | North America | Continent | | 6 | 1 | Oceania | Continent | | 8 | 1 | South America | Continent | +------------+------------------+---------------+--------------+ 7 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a routput identifies ce the seven continents that are directly connected to the root node (planet Earth). T i l The t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-12

MySQL Developer Techniques

Chapter 7: Hierarchical Data

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

3. Locate all the leaf nodes (those children nodes without any children nodes connected to them) using the following SQL statement:

Thi

mysql> SELECT * FROM LocationAdjacencyList leaves -> WHERE NOT EXISTS ( -> SELECT NULL -> FROM LocationAdjacencyList children -> WHERE children.ParentLocationID = leaves.LocationID -> ); +------------+------------------+--------------------+--------------+ | LocationID | ParentLocationID | LocationName | LocationType | +------------+------------------+--------------------+--------------+ | 182 | 23 | British Indian ... | Country | | 243 | 30 | United States ... | Country | | 254 | 32 | Antarctica | Country | | 255 | 32 | French Southern ...| Country | | 256 | 32 | Bouvet Island | Country | | 257 | 32 | Heard Island ... | Country | ... | 6269 | 1682 | Valera | City | | 6270 | 1683 | San Felipe | City | | 6271 | 1684 | Cabimas | City | | 6272 | 1684 | Ciudad Ojeda | City | | 6273 | 1684 | Maraca+¡bo | City | +------------+------------------+--------------------+--------------+ 4085 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ This statement is one of multiple ways to accomplish the e task n of querying for the 4,000 plus leaf 21 table. s d t nodes contained in the LocationAdjacencyList p Stu Come up with at least two additional s ways to accomplish this same task. @ this n a r ฺt use n o o to m ( e an icens r T l et

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-13

MySQL Developer Techniques

Chapter 7: Hierarchical Data

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

4. Use the following query to produce an output that displays the complete hierarchy. Note: This query will produce 24,000 + records and the LIMIT function has been added to prevent the complete record set from being displayed.

Thi

mysql> SELECT t1.LocationName AS Level1, -> t2.LocationName AS level2, t3.LocationName AS level3, -> t4.LocationName AS level4, t5.LocationName AS level5, -> t6.LocationName AS level6 -> FROM LocationAdjacencyList AS t1 -> LEFT JOIN LocationAdjacencyList AS t2 -> ON t2.ParentLocationID = t1.LocationID -> LEFT JOIN LocationAdjacencyList AS t3 -> ON t3.ParentLocationID = t2.LocationID -> LEFT JOIN LocationAdjacencyList AS t4 -> ON t4.ParentLocationID = t3.LocationID -> LEFT JOIN LocationAdjacencyList AS t5 -> ON t5.ParentLocationID = t4.LocationID -> LEFT JOIN LocationAdjacencyList AS t6 -> ON t6.ParentLocationID = t5.LocationID -> LIMIT 45;

e

bl a r fe

s

an r t n

+--------+--------+----------------+--------------------------------+--------------+-------------+ | Level1 | level2 | level3 | level4 | level5 | level6 | +--------+--------+----------------+--------------------------------+--------------+-------------+ | Earth | Africa | Central Africa | Angola | Benguela | Benguela | | Earth | Africa | Central Africa | Angola | Benguela | Lobito | | Earth | Africa | Central Africa | Angola | Huambo | Huambo | | Earth | Africa | Central Africa | Angola | Luanda | Luanda | | Earth | Africa | Central Africa | Angola | Namibe | Namibe | | Earth | Africa | Central Africa | Cameroon | Centre | YaoundT | | Earth | Africa | Central Africa | Cameroon | ExtrOme-Nord | Maroua | ... | Earth | Africa | Central Africa | Sao Tome and Principe | Aqua Grande | Spo TomT | | Earth | Africa | Eastern Africa | British Indian Ocean Territory | NULL | NULL | | Earth | Africa | Eastern Africa | Burundi | Bujumbura | Bujumbura | | Earth | Africa | Eastern Africa | Comoros | Njazidja | Moroni | | Earth | Africa | Eastern Africa | Djibouti | Djibouti | Djibouti | | Earth | Africa | Eastern Africa | Eritrea | Maekel | Asmara | | Earth | Africa | Eastern Africa | Ethiopia | Addis Abeba | Addis Abeba | | Earth | Africa | Eastern Africa | Ethiopia | Amhara | Bahir Dar | +--------+--------+----------------+--------------------------------+--------------+-------------+ 45 rows in set (#.## sec)

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

The first 4,084 records will contain the planet (Earth) in the level1 column with the continents taking the level2 column, the regions taking the level3 column, and so on. The next set of records will switch over with the continents taking the level1 column , the regions taking the level2 column and so on. This process will continue until the cities themselves take the level1 column with NULL's filling in the remaining columns. Use the LIMIT [column #], [# of columns to display] to search different levels of the output remembering that the output will produce 24,000 + records.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-14

MySQL Developer Techniques

7.3.2.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

199

Thi

Chapter 7: Hierarchical Data

Modify Tree Values Using the adjacency list approach, there are issues that must be addressed and dealt with. One of this issues is modifying the data in the list. For example, if there is a need to remove a parent node, such as node B in the diagram, then something has to be done to the child nodes. In this case, node D will take over the "parenting" of the child nodes. This is accomplished in a two step process. First, reassign any of the "parenting" responsibilities from node B to node D:

mysql> UPDATE hierarchy SET Parent='D' -> WHERE Parent='B'; mysql> SELECT * FROM hierarchy; +------+--------+ | Node | Parent | +------+--------+ | A | NULL | | B | A | | C | A | | D | A | | E | D | | F | D | | E | C | | G | C | +------+--------+ 8 rows in set (#.## sec)

A

B

C

ns a r t - G n E

no a s a h n) ideฺ v ฺ m Gu o c ฺ t list: The second step would then be to remove the node from 1 the adjacency n 2 e s pt tud mysql> DELETE FROM hierarchy WHERE s Node='B';S @ this n -> SELECT * FROM hierarchy; a r +------+--------+ A ฺt use n | Node | Parent | o o to +------+--------+ m ( e | A | NULLn | ns | C |TAra | ce C l|i | D et | A | gu|| yEF || DD |

N

| E | C | | G | C | +------+--------+ 7 rows in set (#.## sec)

E

E

e

bl a r fe

D

F

G

D

E

F

For the most part, modifying the data of an adjacency list is a manual multi-step process. However, the most limiting issue associated with adjacency lists is that with pure SQL (no stored procedures or external applications) the end user must determine the level of the hierarchy prior to designing their queries. This limitation can be resolved using other methods of storing data (discussed later), but by far the adjacency list is the least complicated approach to hierarchical data in an SQL environment.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-15

MySQL Developer Techniques

Chapter 7: Hierarchical Data

7.4. Nested Set Structures

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

200

The adjacency list structure is a simple approach to hierarchical data with built-in limitations and performance issues. The deeper the tree, the more issues present themselves. The alternative is the nested set structure. The nested set structure turns the tree inside itself and produces what could be described as a container-like approach to hierarchal data.

A E

B

F

G

C

H

D

e

bl a r fe

T

ns a r t -within this In this example, the top of the tree is the A "container". All paths of the tree are contained n o container. The second level of the tree has three "containers": B, C and D. The B and aof nthe Ctree."containers" each have two containers within them, consisting of what would be the third level s a h 201 ) The means by which the nested set structure is stored within a table involves ฺ is known as a left n idewhat v ฺ and right edge approach. om t Gu c ฺ Node Lnode Rnode 1 n A 2 e s d t A 1 16 sp s Stu @ iB D C B 2 7 n h a t r t e C 8 13 onฺ to1 u2s3 E 4 5 F 6 7 8 9 G 10 11 H 12 13 14 o D 14 (m 15 e n ns a E Tr 3 ice4 l t e y F 5 6 u g G 9 10 hi N H

11

15 16

12

The left and right edges identify the edge of the container. For example, starting from left to right, the left edge of the A "container" is the first edge and is given the identifier 1. The next left edge is in the B "container" and is given the identifier 2. The third left edge is in the E "container" and is given the identifier 3. The next edge reached is the right edge of the E "container" and thus, the next identifier, 4, will be used with that edge. This continues until the farthest right edge is reached (the D "container" is of course taken into account also).

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-16

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

202

Thi

Chapter 7: Hierarchical Data

Even though "containers" provide a better visual when describing nested sets, this process of numbering the left and right edges could have been accomplished using the original tree structure that was seen earlier:

2

B

7

1

A

16

8

C

13

14

D

15

e

3

E

4

5

F

6

9

G

10 11

H

12

s

an r t n

o

an s a h mysql> CREATE TABLE nested (Node CHAR(1), Lnode INT, Rnode INT); ฺ ) ฺvn uide m mysql> INSERT INTO nested VALUES ('A',1,16), ('B',2,7), ('C',8,13), G co nt('H',11,12); ฺ -> ('D',14,15), ('E',3,4), ('F',5,6), 1 ('G',9,10), 2 ts tude pnested; mysql> SELECT Node, Lnode, Rnode FROM s S @ s +------+-------+-------+ i n h t | Node | Lnode | Rnode | tra e ฺ s +------+-------+-------+ n o6 | to u o | A | 1 | m ( n | B | 2 | 7 s |e | C | ran 8 | 13 | e | D t |T 14 |lic 15 | | y Ee | 3 | 4 | gu| F | 5 | 6 |

203

N

bl a r fe

| G | 9 | 10 | | H | 11 | 12 | +------+-------+-------+ 8 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-17

MySQL Developer Techniques

7.4.1.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

204

Chapter 7: Hierarchical Data

Search Tree Depths Once the hierarchical data has been stored in the database using a nested set, it is time to develop a means to query the data. Again, there are multiple activities that may be necessary for the end administrator to accomplish concerning the data. Leaf nodes only There are times when it is beneficial to only view the leaf nodes of the hierarchy. This can be accomplished for nested structures using the following basic query:

mysql> SELECT Node FROM nested -> WHERE Rnode = Lnode + 1; +------+ | Node | +------+ | D | | E | | F | | G | | H | +------+ 5 rows in set (#.## sec)

e

bl a r fe

s

an r t n

T

no a s a h This query displays only those nodes that are leaf nodes (they haven no) child nodes connected to them). eofฺ the d The means to accomplish this is by evaluating the left valuem andฺv the right value node. If there is i u othus mustt G a difference of one value, then there are no child nodesฺc and be a leaf node. 21 den Retrieve a single path s t tu spto display S With the nested set structure, the ability a single path is simplistic in comparison to the @ s i the query used to display the complete hierarchy n below resembles h adjacency list structure. The rquery a 205 t t e with the following changes: parent snode is displayed (rather than the child node) until the query u onforฺ the o reaches the node selected the path to terminate. o t m ( e ns anparent_id.Node r e mysql> SELECT c T li ->t FROM nested AS node_id, e y nested AS parent_id gu -> -> WHERE node_id.Lnode BETWEEN N hi -> parent_id.Lnode AND parent_id.Rnode -> AND node_id.Node = 'E' -> ORDER BY node_id.Lnode; +------+ | Node | +------+ | A | | B | | E | +------+ 3 rows in set (#.## sec)

3

E

4

1

A

16

2

B

7

8

C

13

14

D

15

5

F

6

9

G

10 11

H

12

In this query, the path terminates at the 'E' leaf node due to it being the node identified in the WHERE clause. _________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-18

MySQL Developer Techniques

7.4.2.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

206

Chapter 7: Hierarchical Data

Display Depth With the adjacency list model, it was important for the developer to know in advance the depths of the hierarchy. This of course is not the most convenient process especially with data that may be changing on a constant basis. With the nested set model, the query can be designed to determine the depths without the developer having foreknowledge of the data. The following SQL demonstrates how this can be accomplished:

mysql> SELECT node_id.Node, (COUNT(parent_id.Node) - 1) AS level -> FROM nested AS node_id, -> nested AS parent_id -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode -> GROUP BY node_id.Node -> ORDER BY node_id.Lnode; +------+-------+ Level 0 | Node | level | A +------+-------+ | A | 0 | Level 1 | B | 1 | C B | E | 2 | | F | 2 | | C | 1 | Level 2 | G | 2 | G F E | H | 2 | | D | 1 | +------+-------+ 8 rows in set (#.## sec)

e

bl a r fe

s

aD n r t n

T

no a s a H h ) ฺ ฺvn uide m co nt G ฺ 1 s2 tude t p Indenting levels s sS @ iREPEAT, indentation can be added to give a visual nCONCAT tand Using SQL string functions like 207 h a r t e representation of the depth s onฺleveltojustucreated: o (m nse mysql> SELECT CONCAT(REPEAT('>', COUNT(parent_id.Node) - 1), node_id.Node) n a -> ASr 'Node Level' e lic AS node_id, ->t T FROM nested e nested AS parent_id uy -> g -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode -> GROUP BY node_id.Node hi N -> ORDER BY node_id.Lnode; +------------+ | Node Level | +------------+ | A | | >B | | >>E | | >>F | | >C | | >>G | | >>H | | >D | +------------+ 8 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-19

MySQL Developer Techniques

7.4.3.

Displaying a specific branch

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

208

Thi

Chapter 7: Hierarchical Data

There are times when it is necessary to display a specific branch of the hierarchical tree. For example, it may be necessary to only display the branch that begins with the 'B' node. The limitation to making this happen with the existing SQL that has been demonstrated is that any restrictions applied will cause a corruption of the results. Adding a sub-query can prevent the corruption of data when restrictions are placed on the output of the query. The following SQL demonstrates how this can work:

A

E

B

C

D

F

G

H

mysql> SELECT node_id.Node, -> (COUNT(parent_id.Node) - (branch.depth + 1)) AS depth -> FROM nested AS node_id, nested AS parent_id, nested AS branch_parent, -> ( SELECT node_id.Node, (COUNT(parent_id.Node) - 1) AS depth -> FROM nested AS node_id, nested AS parent_id -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode -> AND node_id.Node = 'B' -> GROUP BY node_id.Node -> ORDER BY node_id.Lnode -> ) AS branch -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode -> AND node_id.Lnode BETWEEN branch_parent.Lnode AND branch_parent.Rnode -> AND branch_parent.Node = branch.Node -> GROUP BY node_id.Node 209 -> ORDER BY node_id.Lnode; +------+-------+ | Node | depth | +------+-------+ | B | 0 | | E | 1 | | F | 1 | +------+-------+ 3 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-20

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

A

Displaying a specific sub-level Similar to displaying a specific branch in a hierarchical tree, there may be times that only the values in a specific sub-level need to be displayed. For example, it may be necessary to only display the nodes directly connected to node 'B'. Using the query that was developed to show a single branch, a HAVING clause can be added to accomplish this task:

210

Thi

Chapter 7: Hierarchical Data

E

B

C

D

F

G

H

mysql> SELECT node_id.Node, -> (COUNT(parent_id.Node) - (branch.depth + 1)) AS depth -> FROM nested AS node_id, nested AS parent_id, nested AS branch_parent, -> ( SELECT node_id.Node, (COUNT(parent_id.Node) - 1) AS depth -> FROM nested AS node_id, nested AS parent_id -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode -> AND node_id.Node = 'B' -> GROUP BY node_id.Node -> ORDER BY node_id.Lnode -> ) AS branch -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode -> AND node_id.Lnode BETWEEN branch_parent.Lnode AND branch_parent.Rnode -> AND branch_parent.Node = branch.Node -> GROUP BY node_id.Node -> HAVING depth = 1 211 -> ORDER BY node_id.Lnode; +------+-------+ | Node | depth | +------+-------+ | E | 1 | | F | 1 | +------+-------+ 2 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-21

MySQL Developer Techniques

7.4.4.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

212

Thi

Chapter 7: Hierarchical Data

Inserting Nodes Adding nodes to a nested tree hierarchy is an action that must be addressed due to the likelihood of it happening in a real-world scenario. There are specific steps that must be taken to ensure that the hierarchy is maintained properly. These steps could be turned into a stored procedure; however, for the purpose of this class the steps will be presented individually. The following SQL demonstrates how a node, in this case node 'I', will be added to the first sub-level directly connected the root node 'A' and to the right of node 'B':

A

B

E

I

F

C

D

G

H

mysql> LOCK TABLE nested WRITE; mysql> SELECT @Current_right := Rnode FROM nested WHERE Node = 'B'; +------------------------+ | @Current_right := Rnode | +------------------------+ | 7 | +------------------------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h mysql> UPDATE nested SET Rnode = Rnode + 2 WHERE Rnode n > )@Current_right; eฺ v ฺ d i u om mysql> UPDATE nested SET Lnode = Lnode + 2 WHERE Lnode >G @Current_right; c ฺ t 21 den s t mysql> INSERT INTO nested(Node, Lnode, Rnode) p Stu -> VALUES ('I', @Current_rights+ 1, @Current_right + 2); @ s i n h tra se t mysql> UNLOCK TABLES; ฺ n o to u o m n ( ense a r T lic t e guy

N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-22

MySQL Developer Techniques

Of course, it is best to check to make sure that these steps actually performed the required action. This can be accomplished using the indented query presented earlier:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

213

Thi

Chapter 7: Hierarchical Data

mysql> SELECT CONCAT(REPEAT('>', COUNT(parent_id.Node) - 1), node_id.Node) -> AS 'Node Level' FROM nested AS node_id, nested AS parent_id -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode -> GROUP BY node_id.Node ORDER BY node_id.Lnode; +------------+ | Node Level | A +------------+ | A | | >B | | >>E | | >>F | B I C | >I | | >C | | >>G | | >>H | E F G | >D | +------------+ 9 rows in set (#.## sec)

D

e

bl a r fe

ns a r t nH

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-23

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

214

Thi

Chapter 7: Hierarchical Data

A

Inserting an only child node There are times when a child node gets promoted to a parent node by having a child node of its own added to the hierarchy. For example, our newly added node 'I' from the previous step has been given the responsibility of parenting node 'J'. The previous update steps would not address this issue, but with a simple modification of those steps it can easily be accomplished. The following SQL demonstrates how this would be handled:

E

B

I

C

D

F

J

G

H

mysql> LOCK TABLE nested WRITE; mysql> SELECT @Current_left := Lnode FROM nested WHERE Node = 'I'; +------------------------+ | @Current_left := Lnode | +------------------------+ | 8 | +------------------------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

no a as h mysql> UPDATE nested SET Lnode = Lnode + 2 WHERE Lnode > )@Current_left; n ideฺ v ฺ u mysql> INSERT INTO nested(Node, Lnode, Rnode) om G c ฺ t VALUES('J', @Current_left + 1, @Current_left 21 + d2);en s t mysql>UNLOCK TABLES; sp s Stu @ i n h a t r t e Again, these steps accomplishing 215 s should be verified with the following SQL: onฺ totheutask o (m nse mysql> SELECTnCONCAT(REPEAT('>', COUNT(parent_id.Node) - 1), node_id.Node) a e -> ASr 'Node Level' FROM nested AS node_id, nested AS parent_id c T li ->t WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode e y -> GROUP BY node_id.Node ORDER BY node_id.Lnode; u Ng+------------+ mysql> UPDATE nested SET Rnode = Rnode + 2 WHERE Rnode > @Current_left;

| Node Level | +------------+ | A | | >B | | >>E | | >>F | | >I | | >>J | | >C | | >>G | | >>H | | >D | +------------+ 10 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-24

MySQL Developer Techniques

7.4.5.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

216

Thi

Chapter 7: Hierarchical Data

Deleting Nodes Once a node is in place with the nested set structure, deleting that node can have negative effects on the hierarchy if the other nodes (there left and right values) are not modified as well. The node function (parent or child) will determine the process that is necessary for deleting the node. Deleting a branch Deleting a complete branch (or a single child node) is probably the easiest form of deletion associated with the nest set structure. For example, node 'C' is a child of node 'A' and can be deleted from the hierarchy along with any child nodes. The SQL associated with making this happen is the exact opposite of an insert of a single node (utilizing the same basic SQL though):

e

bl a r fe

ns a r t mysql> SELECT @Current_left := Lnode AS CL, @Current_right := Rnode AS CR, n o -> @Current_width := Rnode - Lnode + 1 AS CW FROM nested WHERE n a Node = 'C'; +------+------+------+ s a | CL | CR | CW | h ) +------+------+------+ n ideฺ v ฺ | 12 | 17 | 6 | m Gu +------+------+------+ o c ฺ t 1 row in set (#.## sec) 1 n 2 e s t ud pBETWEEN t@Current_left s mysql> DELETE FROM nested WHERE Lnode AND @Current_right; S @ s i an= Rnode th- @Current_width WHERE Rnode > r mysql> UPDATE nested SET ฺRnode t e @Current_right; on to us o (m SETseLnode = Lnode - @Current_width WHERE Lnode > mysql> UPDATEnnested n a r @Current_right; ce T i l t ye UNLOCK TABLES; gumysql> mysql> LOCK TABLE nested WRITE;

N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-25

MySQL Developer Techniques The SQL steps presented handled the deletion of the entire branch. Like updates, it is important to review the data to see if the delete actually took place. The standard indentation SQL that has been used previously can be used again:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

217

Thi

Chapter 7: Hierarchical Data

mysql> SELECT CONCAT(REPEAT('>', COUNT(parent_id.Node) - 1), node_id.Node) -> AS 'Node Level' FROM nested AS node_id, nested AS parent_id -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode -> GROUP BY node_id.Node ORDER BY node_id.Lnode; +------------+ | Node Level | +------------+ | A | | >B | | >>E | | >>F | | >I | | >>J | | >D | +------------+ 7 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-26

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

218

Thi

Chapter 7: Hierarchical Data A

Deleting a parent without deleting the children The more complex deletion process is when a parent node is to be deleted, but the children should remain in the hierarchy. Now of course, the easiest way to handle a deletion of a parent is simply keeping the parent in place but identifying it as an empty parent until a new parent can take its place (such as a management position being vacated and kept in the hierarchy until a new manager can be placed in that node).

E

B

I

F

J

D

mysql> LOCK TABLE nested WRITE; mysql> SELECT @Current_left := Lnode, @Current_right := Rnode FROM nested -> WHERE Node='I'; +------------------------+-------------------------+ | @Current_left := Lnode | @Current_right := Rnode | +------------------------+-------------------------+ | 8 | 11 | +------------------------+-------------------------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

no a as mysql> UPDATE nested SET Rnode = Rnode - 1, Lnode = Lnode) -h1 n ideฺ -> WHERE Lnode BETWEEN @Current_left AND @Current_right; v ฺ u G comRnode ฺ t mysql> UPDATE nested SET Rnode = Rnode - 21WHERE > @Current_right; n 2 e s d t tu Lnode > @Current_right; mysql> UPDATE nested SET Lnode = Lnode sp - s2 SWHERE @ i n h a t mysql> UNLOCK TABLES; r t e onฺ to us o 219 Again, these steps (maccomplishing e the task should be verified with the following SQL: s n n a r ce T i l t mysql> SELECT CONCAT(REPEAT('>', COUNT(parent_id.Node) - 1), node_id.Node) e-> AS 'Node Level' FROM nested y AS node_id, nested AS parent_id u Ng -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode mysql> DELETE FROM nested WHERE Lnode = @Current_left;

-> GROUP BY node_id.Node ORDER BY node_id.Lnode; +------------+ | Node Level | +------------+ | A | | >B | | >>E | | >>F | | >J | | >D | +------------+ 6 rows in set (0.00 sec)

A

B

E

J

D

F

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-27

MySQL Developer Techniques

7.5.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

220

Thi

Chapter 7: Hierarchical Data

Path Enumeration A path enumeration hierarchy model stores the unique path that is created from the root node to every node in the tree. This path is stored as a string by concatenating either the edges (as in the nested set hierarchy model) or the keys (as in the adjacency list hierarchy model) of the nodes in the path. Searches are completed using string functions that parse the stored path values. The following table represents an example of using the keys to identify the path A from root to the node that is represented: Node

Path

A

A

B

A/B

C

A/C

D

A/D

E

A/B/E

F

A/B/F

G

A/C/G

E

B

C

D

F

G

H

e

bl a r fe

s

an r t n

o n a H A/C/H s a h The following SQL statements represents how the associated database forฺ this table would be n) table 221 e v ฺ d i created and loaded with the hierarchical values: om t Gu c ฺ mysql> CREATE TABLE hier_path (Node CHAR(1), n 21 PathdVARCHAR(15)); e s t p Stu('B','A/B'), ('C','A/C'), mysql> INSERT INTO hier_path VALUESs('A','A'), @ -> ('D','A/D'), ('E','A/B/E'), ('F','A/B/F'), ('G','A/C/G'), ('H','A/C/H'); is n h a t r t e mysql> SELECT * FROM hier_path; onฺ to us o +------+-------+ (m nse | Node | Path | n +------+-------+ a r |lice | A t |TA | u|| yBC e || A/B A/C g N | D | A/D || | E | A/B/E | | F | A/B/F | | G | A/C/G | | H | A/C/H | +------+-------+

The string column (in this case Path) can not be greater than the number of nodes in the given table and thus is given a variable character identification that can not exceed 15 characters (8 for the node ID and 7 for the separator character) in this table.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-28

MySQL Developer Techniques

7.5.1.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

222

Chapter 7: Hierarchical Data

Search the depths of the path

Level 1

The path enumeration model is probably the simplest approach to storing hierarchical data when it is time to search the depths of the hierarchical tree.

Level 2

A

Determining the depth of the tree The maximum depth of the tree can be obtained by evaluating the longest path string to determine how many depth levels are contained in the hierarchy. For example, using the hier_path table, the following SQL statement determines the maximum levels of depth that the hierarchy contains:

Level 3

E

B

C

D

F

G

H

mysql> SELECT MAX(LENGTH(REPLACE(Path,'/',''))/LENGTH(Node)) AS Depth -> FROM hier_path; +------------+ | Depth | +------------+ | 3.00000000 | +------------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h This output displays the maximum depth that the hierarchical tree created by taking the length n)Thiscontains eฺcan v ฺ d i value. value then be used in of the Path value divided by the fixed length of the Node m reports umost often required. o G conjunction with other SQL functions to produce the hierarchical c ntthe first level being identified as 1 21ฺ startsdewith s Note: that the depths for a path enumerationthierarchy tu the first level as 0. sp which which is different than the nested tree approach identified S @ s i n h a t r t e onฺ to us o (m nse n a r ce T i l t uye

g

N Thi

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-29

MySQL Developer Techniques Display the depth for each node

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

223

Thi

Chapter 7: Hierarchical Data

Similar to determining the maximum depth of a tree, each nodes depth level can be displayed by utilizing the same formula: mysql> SELECT Node, LENGTH(REPLACE(Path,'/',''))/LENGTH(Node) -> AS Depth_Level FROM hier_path; +------+-------------+ | Node | Depth_Level | +------+-------------+ | A | 1.0000 | | B | 2.0000 | | C | 2.0000 | | D | 2.0000 | | E | 3.0000 | | F | 3.0000 | | G | 3.0000 | | H | 3.0000 | +------+-------------+ 8 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v Level 1 ฺ A om t Gu c ฺ 1 n s2 tude t p s Level S2 @ s i B C n h a t r t e onฺ to us o Level 3 (m nse n E F G a r e c T i l et

Each node and their location in the depth levels of the hierarchy is displayed. The root node will be the only node on the first level.

D

H

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-30

MySQL Developer Techniques

7.5.2. 224

Chapter 7: Hierarchical Data

Retrieving hierarchical data from the paths In addition to determining the levels of the hierarchy, the ability to locate and display specific nodes based on a set criteria can be crucial to the success of a hierarchical model chosen.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Retrieve a single path

Thi

Due to the path being stored in the path enumeration hierarchy, retrieving a path for a single node is as easy as producing a SELECT statement that simply searches for the row associated with the node requested: mysql> SELECT * FROM hier_path WHERE Node = 'H'; +------+--------+ | Node | Path | +------+--------+ | H | A/C/H | +------+--------+ 1 row in set (#.## sec)

225

Locating children of a node

A

D

C

B

e

E

G

F

H

bl a r fe

s tran

n o n

There are times when it is necessary to report only the children of a specific node. The following SQL statement will search for all children associated with the B node:

a s a mysql> SELECT * FROM hier_path WHERE Path LIKE '%/B/%'; ) h n ideฺ +------+--------+ v ฺ | Node | Path | om t Gu +------+--------+ c ฺ | E | A/B/E | 21 den s t | F | A/B/F | sp s Stu +------+--------+ @ i 2 rows in set (#.## sec) n h a t r t E e ฺ s n u o o to m ( e an icens r T l t e y gu

A

B

C

D

F

G

H

N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-31

MySQL Developer Techniques

Chapter 7: Hierarchical Data

7.6. Summary In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

226

Thi

•

Define graphs, trees and hierarchies

•

Create and modify an adjacency list structure

•

Create and modify nested set structures

•

Create and modify path enumeration structures

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

7-32

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 8

e

s

bl a r fe

an r t n

no ADVANCED INDEX a s a h n) ideฺ v STRUCTURES ฺ u om

uy g N

ฺc nt G 1 2 e pts Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

8. ADVANCED INDEX STRUCTURES 8.1. Overview

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

228

Thi

This chapter provides the knowledge and skills necessary to utilize advanced index structures. At the completion of this chapter, you will be able to: •

Describe MySQL index types

•

Optimize end of field searches

•

Create FULLTEXT indexes for faster text searches

•

Simulate function based indexes in MySQL

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-1

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

8.2. MySQL Index Types

Thi

229

MySQL offers a number of indexing types to use with the server; however, some storage engines will limit the type of indexing that can be used and can provide additional indexing types for use with the specific storage engine. The following are the most common index types used by the MySQL server: B-Tree This widely used data structure emulates a tree structure with a set of linked nodes. There are four common terms for nodes that tree indexes can contain: •

Root node – This is the starting node for the tree, all nodes within a tree will be connected to this node, either directly or indirectly.

•

Parent node – This is a node that is connected to a node above it and contains nodes below it. This is sometimes referred to as an internal node.

•

Child node – This a node that is connected to a parent node. A child node must have at least one parent node.

e

bl a r fe

Leaf node – This is a node that could be referred to as a child node but with one additional feature; leaf nodes have no child nodes connected to them. Due to each node having zero or more child nodes, the graphical representation looks like a tree growing down, not up. The B-tree indexing type adds an additional requirement that keeps the tree balanced (there is some belief that the ‘B’ in B-Tree stands for balanced). A B-tree is kept balanced by requiring that all leaf nodes are at the same depth. This depth will increase slowly as elements are added to the tree, but an increase in the overall depth is infrequent, and results in all leaf nodes being one more hop further removed from the root. By maximizing the number of child nodes for each parent node, the depth of the tree decreases, balancing occurs less often, and efficiency increases. All nodes have the same size (ie the "block" structure of the index). This is important as BTrees are optimized for minimizing disk reads. •

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t Root node sp s Stu (non-leaf) @ i n Pointers to h tra se t ฺ child leaf n o 20to u nodes o 40 60 m ( nse n a ice r T l et

uy g N

1

Keys 2-19

19

21

Keys 22-39

39

41

Keys 42-59

Leaf nodes

59

Pointers to data

Key values

Actual data

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

230

Thi

Chapter 8: Advanced Index Structures

B+TREE Index nodes The B+Tree index is a data (i-nodes) structure to store vast amounts of information and 3 5 is the next level B-Tree. Typically B+Trees are used Leaf Key nodes to store amounts of data that values will not fit in main system memory. To do this, 3 2 1 5 4 7 6 secondary storage (usually disk) is used to store the Pointers to leaf nodes of the tree. Only data the internal nodes of the tree Links allow for rapid in-order are stored in computer traversal Actual data memory. In a B+Tree the leaf nodes are the only ones that actually store data items. All other nodes are called index nodes or i-nodes and simply store ''guide'' values which allow us to traverse the tree structure from the root down and arrive at the leaf node containing the data item we seek. Because disk I/O is very slow in comparison to memory access these leaf nodes store more than one data item each. In fact, the data structure will perform best when the size of the leaf nodes closely approximates the size of a disk sector under most operating systems. Thus, when a search is performed against a B+Tree (by traversing from the root node down to the proper data node) a read must take place against the data node from the disk to search its contents. Another way to improve the speed of a query operation is to keep a memory cache of recently read leaf nodes. A B+Tree is a rooted search tree satisfying the following properties: • All paths from root to leaf are of the same length. • The index node has ... o at least 2 children if not a leaf. o between 0 and (n–1) values (index entries), if a leaf. • All other nodes are at least half-full, i.e. they have ... o between [n/2] and n children, if not a leaf. o between [(n–1)/2] and n–1 values (index entries) if a leaf. • The fanout n (width of the index) depends on the size of the search-key and the size of disk blocks. Advantages of B Tree and B+Tree indexes: • Automatically reorganizes itself with small, local, changes, in the face of insertions and deletions. • Reorganization of entire file is not required to maintain performance. Disadvantage of B Tree and B+Tree indexes: • Extra insertion and deletion overhead, space overhead.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-3

MySQL Developer Techniques

HASH A hash is simply a key/value pair and a hash index is a collection of those key/value pairs. The hash index works efficiently by taking a lookup value (the value portion of the key/value pair), obtaining the specific key associated with the lookup value, then transforming the key using a hash function into a hash, and then using the hash number with the associated hash table to locate the desired value.

231

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 8: Advanced Index Structures

Keys

Indexes

Records (Rows)

0000 Robert Freed

0001

Pierre Debok

+1-213-555-4567

0002 B20A

e

bl a r Robert Freed +1-913-555-6786 B20C fe s n ra t n FAB8 o n Gunter Greese a FAB9 s a h FABA n) ideฺ v ฺ om t Gu c Hash indexes have the following characteristics: ฺ n 21 that use e s • They are used only for equality comparisons the = or operators (but are very fast). d t tuas < that find a range of values. spoperators They are not used for comparison such S @ s i to speed up ORDER BY operations (this type of index na hash index h • The optimizer cannot use a t r t e cannot be used n to ฺsearch for the s next entry in order). u o o o t approximately how many rows there are between two values (this • MySQL determine (miscannot e s statistic sometimes n n used by the optimizer to decide which index to use). a r e c T Only whole li keys can be used to search for a row. (With a B-tree index, any leftmost prefix of t • the e key can be used to find rows). y u Pierre Debok

g

N Thi

Gunter Greese +1-315-555-2314

•

B20B

A HASH index works within a small space and is extremely fast; however, can be very slow with many non-unique values.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-4

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

8.3. FULLTEXT Index

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

232

Thi

The MySQL server supports full-text indexing and searching which is useful for word searches in text and searches on several columns. A full-text index in MySQL is an index of type FULLTEXT and can be used only with MyISAM tables. Creating a FULLTEXT Index Full-text indexes can be created for CHAR, VARCHAR, or TEXT columns with CREATE TABLE. A FULLTEXT index may also be added later using ALTER TABLE or CREATE INDEX. In large datasets, it is much faster to load the data into a table that has no FULLTEXT index and then create the index subsequent to the load. The following syntax is an example of a FULLTEXT index being created at table creation time: mysql> -> -> -> -> -> -> ->

CREATE TABLE film_text ( film_id int(10) NOT NULL default '0', title varchar(255) NOT NULL default '', description text, category_id int(10) NOT NULL default '0', PRIMARY KEY (film_id), FULLTEXT KEY title (title,description) ) ENGINE=MyISAM;

e

bl a r fe

s

an r t n

no a sthe table has already been a The following syntax is an example of a FULLTEXT index being addedhafter created: n) ideฺ v ฺ om t Gu c ฺ mysql> ALTER TABLE film_text 21 den -> ADD FULLTEXT KEY title (title,description), s t -> engine=MyISAM; sp s Stu @ i n h a t r t e onฺ to usFULLTEXT is a B-Tree o (m nse n a The FULLTEXT as a kind of normalized, condensed version of the actual text and created as a B-Tree r indexlisicstored eof column T index (words are used instead values). Words that appear frequently in the data losing their effectiveness as t e y search terms are removed and other words grouped together. In addition, stopwords (words that are pre-defined by userver as common) are removed. Whenarea record g the is inserted into a MyISAM table containing a FULLTEXT index, N the data for the indexed fields are analyzed and split into “words”.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-5

MySQL Developer Techniques

8.3.1.

FULLTEXT Search Functions Full-text searching is performed using MATCH() ... AGAINST syntax. MATCH() takes a commaseparated list that names the columns to be searched. AGAINST takes a string to search for, and an optional modifier that indicates what type of search to perform. The search string must be a literal string, not a variable or a column name. There are three types of full-text searches:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

233

Thi

Chapter 8: Advanced Index Structures

• 234

Boolean - A boolean search interprets the search string using the rules of a special query language. The string contains the words to search for. It can also contain operators that specify requirements such that a word must be present or absent in matching rows, or that it should be weighted higher or lower than usual. Common words such as “some” or “then” are stopwords and do not match if present in the search string. The IN BOOLEAN MODE modifier specifies a boolean search. The following example demonstrates a FULLTEXT boolean search:

235 mysql> SELECT id, title, body FROM articles WHERE MATCH (title,body) -> AGAINST ('+MySQL -YourSQL' IN BOOLEAN MODE); +----+-----------------------+-------------------------------------+ | id | title | body | +----+-----------------------+-------------------------------------+ | 1 | MySQL Tutorial | DBMS stands for DataBase ... | | 2 | How To Use MySQL Well | After you went through a ... | | 3 | Optimizing MySQL | In this tutorial we will show ... | | 4 | 1001 MySQL Tricks | 1. Never run mysqld as root. 2. ... | | 6 | MySQL Security | When configured properly, MySQL ... | +----+-----------------------+-------------------------------------+

e

bl a r fe

s

an r t n

236

no a s a h n) ideฺ v ฺ The + and - operators indicate that a word is required present respectively, for a mto be contain u ortheabsent, G match to occur. Thus, this query retrieves allฺc theorows that word “MySQL” but that t n do not contain the word “YourSQL”. s21 e d t p Stu o ft_boolean_syntax - The s ft_boolean_syntax system variable displays the list of @ s i operators supportedn by booleanh full-text searches performed using IN BOOLEAN MODE. t is + ->) and less than ( SELECT user_name AS user, entered', p s S > word as 'dictionary word','normal' as 'Output' @ s i n JOIN passwords > FROM words INNER ON password=word h a t r t -> UNION e ฺ n uASs user, REVERSE(word), word,'reverse' AS 'Output' -> SELECTouser_name o o t -> FROM words INNER (m nse JOIN passwords ON password=reverse_word; +---------+------------------+-----------------+---------+ n a ic| epassword entered | dictionary word | Output | | ruser T l t +---------+------------------+-----------------+---------+ e y | kgreer | homework | homework | normal | gu | rspeer | button | button | normal | N | dakin | ssob | boss | reverse | hi Step 3: Create the query to test the passwords

| llawson | enots | stone | reverse | +---------+------------------+-----------------+---------+ 4 rows in set (#.## sec)

The first SELECT will test to see if the password matches exactly a word in the words table. If there is a match the user name is displayed, along with their password entered, the dictionary word that was matched and an output describing it as a normal match. The second select, which is joined to the first SELECT with the UNION command. again displays the user name, but this time checks to see if the password is a dictionary word reversed. If so, the password entered is displayed, the "normal" dictionary word is displayed and the output reflects that it is a reverse password of a dictionary word.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-9

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

8.5. Optimizing End of Field Searches

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

243

243

Indexes are built with the assumption that words go from left to right. For the majority of fields indexed, this is true. However, there are times when this is not true and using standard index practice will not improve searches that need to search the right most portion of a field. For example, the following list contains e-mail addresses for customers in a typical database mailing list: [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] ... If there was a need to contact all customers that have an account on the sun.com mail server, a typical search would involve the following SQL statement:

e

bl a r fe

s

an r t n

T

no a s this specific search is In this case, even if the email column is indexed, that index is not useful a because blocked from using the index lookup with the % wild card at the front)ofh the search. n ideฺ v ฺ Mirror image om totwork Gufrom right to left. Until such a 245 This of course could be resolved if the index could1 beฺc designedn feature can be added by an industrius contributor s2to thetuMySQL de code, there is a very easy work around t p to correct this problem: the reverse function. s S @ s i n h The reverse function takes a string creates t a mirror image of that string. By creating a new column tra and e ฺ in our table (let's call it n rev_email) and then s o to u running the following SQL we fill this new column with a mirror image of theo e-mail field: (m customers se SET rev_email=REVERSE(email); n mysql> UPDATE n a r e T the data shown lic above, this UPDATE query will fill the following data into the rev_email column: t Using e uy g [email protected] N i [email protected] h mysql> SELECT email FROM customers WHERE email LIKE '%sun.com;

[email protected] [email protected] [email protected] [email protected] [email protected] [email protected] ...

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-10

MySQL Developer Techniques

Once the data has been reversed, an index can be applied to this column and the original search could then be reissued:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

246

Thi

Chapter 8: Advanced Index Structures

mysql> SELECT email FROM customers WHERE rev_email -> LIKE 'moc.nus%' +-----------------------+ | email | +-----------------------+ | [email protected] | | [email protected] | +-----------------------+

Final thoughts 247

After the rev_email column has been added to the table, keeping this column updated is a job for triggers. Both an update trigger (to catch changes to e-mail addresses) and an insert trigger (for new data being added to the table) will be required:

e

bl a r mysql> CREATE TRIGGER Customers_BU_EmailReverse BEFORE UPDATE ON customersfe -> FOR EACH ROW SET NEW.rev_email = REVERSE(NEW.email); ns a r t n mysql> CREATE TRIGGER Customers_BI_EmailReverse BEFORE INSERT o ON customers -> FOR EACH ROW SET NEW.rev_email = REVERSE(NEW.email); an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-11

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

Further Practice Lab 8-A

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this lab you will use the knowledge you have learned in this chapter to create an SQL statement that uses reverse logic on the Country table in the world database.

Thi

Step 1: Examine the indexes already present in the Country table Utilizing the world database, list the indexes currently in place for the Country table. Step 2: Create and test an index on the Name column in the Country table Create an index on the Name column of the Country table. Obtain information about how MySQL would execute the following SELECT statement:

e

bl a r fe

mysql> SELECT Name, Population FROM Country WHERE Name LIKE '%land'; Obtain information about how MySQL would execute the following SELECT statement:

ns a r t mysql> SELECT Name, Population FROM Country WHERE Name LIKE- 'S%'; on n a Which SELECT statement would perform better? Why? s a h ฺ ) __________________________________________________________________________________ n ฺv uide __________________________________________________________________________________ m o ฺc nt G 1 __________________________________________________________________________________ 2 e pts Stud s Step 3: Create and update a reverse based @column is on the Name column of the world database n h a t r ฺtto the Country Add a new indexed column se table called NameReverse that allows for 52 characters n u o and is located after o the Name column. to m ( e n columnewith Fill this new nsa mirror image of the Name column using the REVERSE function on all the a r c T t records lini the table. ecurrent

uy Step 4: Search for a string at the "end" of a column g N

Obtain information about how MySQL would execute the following SELECT statement (similar to the poor performing SELECT statement presented earlier): mysql> SELECT Name, Population FROM Country -> WHERE NameReverse LIKE 'dnal%'; The server is now able to utilize the index due the lead % being removed and the result is 14 rows having to be evaluated versus a fill table scan (293 rows).

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-12

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

Step 5: Create triggers to support long-term maintenance of the new column

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

After the NameReverse column has been added to the Country table, keeping this column updated is a job for triggers. Create both an update trigger (to catch changes to e-mail addresses) and an insert trigger (for new data being added to the table).

Thi

Step 6: Test the alteration and addition of new rows With the triggers in place to handle both updates and additions to the Country table, it is best to test it to make sure that the triggers will work as expected. Test the alteration of existing data with the following SQL statements: mysql> UPDATE Country SET Name='Beigeland' WHERE Name='Greenland'; mysql> SELECT Name, NameReverse FROM Country WHERE NameReverse LIKE 'dnal%';

Now test the triggers that manage new records being inserted by entering the following SQL statements into the mysql client:

e

bl a r fe

s

an r t n

mysql> INSERT INTO Country (Code, Name) VALUES ('SUN','Sunland'); mysql> SELECT Name, NameReverse FROM Country WHERE NameReverse LIKE 'dnal%';

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-13

MySQL Developer Techniques

8.6.

Summary In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

248

Thi

Chapter 8: Advanced Index Structures

•

Describe MySQL index types

•

Optimize end of field searches

•

Create FULLTEXT indexes for faster text searches

•

Simulate function based indexes in MySQL

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

8-14

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 9

e

s

bl a r fe

an r t n

no LOCKING LEVELS a as

uy g N

h ฺ ) n ฺv uide m o ฺc nt G 1 2 e pts Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

9 9.1

LOCKING LEVELS Overview This chapter provides the knowledge and skills necessary to explain locking levels and create queries that compensate for the locking level used. At the completion of this chapter, you will be able to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

250

Thi

Chapter 9:Locking Levels

•

Explain locking fundamentals

•

Utilize explicit table locks

•

Explain the different locking techniques of the storage engines

•

Explain how InnoDB's row-locking and next-key-locking work

•

Describe the most common issues associated with locking

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-1

MySQL Developer Techniques

9.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

251

Chapter 9:Locking Levels

Locks Locking is a mechanism that prevents problems from occurring with simultaneous data access by multiple clients. Locks are managed by the server: It places a lock on data on behalf of one client to restrict access by other clients to the data until the lock has been released. The lock allows access to data by the client that holds the lock, but places limitations on what operations can be done by other clients that are contending for access. The effect of the locking mechanism is to serialize access to data so that when multiple clients want to perform conflicting operations, each must wait its turn. When discussing locks it is important to see them as a synchronization mechanism for enforcing limits on access to a resource in an environment where there are many threads of execution. Locks are one way of enforcing concurrency control policies and maintaining data integrity. There are two types of locks:

252

•

Shared locks – A shared lock means that many clients can lock the same data (for reading only), as the lock is shared between the clients. This could be the case when the client wants to read data and ensure that the data is not changed by anyone else.

•

Exclusive locks – An exclusive lock means that only the client locking the data can access it. This type of lock is often used when the client is changing data.

Acquiring locks

e

bl a r fe

s

an r t n

T

no a • Implicit locks – For a client that does nothing special to acquireslocks, the MySQL server a safely. For example, implicitly acquires locks as necessary to process the client's h statements ) n engineidwill when a client reads or writes rows from a table, the storage eฺ implicitly use locks to v ฺ avoid conflicts. om t Gu c ฺ n • Explicit locks – In some cases, if implicit iseinsufficient for a client's purposes, it can 21atlocking s d t manage locks explicitly by acquiring them the beginning of an operation and releasing them at u p t sbe necessary Swhen a client needs to perform an operation that the end. Explicit locking may @ s i n h spans multiple statements and that must not be interrupted by other clients. a et r t ฺ Releasing locks on to us 253 o A session can(m or serelease locks only for itself. One session cannot acquire locks for another n acquire n sessionra or release locks held by another session. Just like acquiring locks, there are explicit and implicit e Tto releasellocks ic that have been acquired: t ways e uy • Explicit release - The UNLOCK TABLES statement explicitly releases any table locks held by g the current thread. hi N When discussing locks, there are two locking acquisitional types that also need to be understood:

•

Implicit release - There are multiple events that cause the MySQL server to release a lock that has been created: o o o

LOCK TABLES releases any table locks currently held by the thread before acquiring new locks. Beginning a transaction (for example, with START TRANSACTION) implicitly performs an UNLOCK TABLES. If a client connection drops, the server releases table locks held by the client. If the client reconnects, the locks will no longer be in effect.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-2

MySQL Developer Techniques Locks granularity

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

254

Thi

Chapter 9:Locking Levels

In MySQL, there are two different levels of locking granularity implemented by different storage engines: •

Table locks – This is when a table as a whole is locked on an all-or-nothing basis.

•

Row locks – This is when an individual row is locked from the other clients. Due to the complexity of performing this locking granularity it is the most difficult one to implement; however, it provides a high level of concurrency as only a minimal resource (a row) is locked.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-3

MySQL Developer Techniques

9.3

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

255

Thi

Chapter 9:Locking Levels

Explicit Locks When discussing explicit locks, there are two specific types of locks that need to be reviewed: Table locks and Row locks. Acquisition of explicit locks can be advantageous in certain situations: • Multiple statement update - An implicit lock lasts for the duration of a single query only, which is unsuitable should you want to perform a multiple-statement update that requires no interference by other clients. To handle this, explicit locks can be acquired, which remains in effect until released. Other clients cannot modify tables or rows that are locked. •

Grouped statements - Explicit locking can improve performance for multiple statements executed as a group while the lock is in effect. o First, less work is required by the server to acquire and release locks because it need not do so for each statement. It simply acquires all needed locks at the beginning of the operation, and releases them at the end. o

Second, for statements that modify data, index flushing is reduced. For example, if multiple INSERT statements are executed using implicit locking, index flushing occurs following each statement. If the table is locked explicitly and then all the inserts are performed, index flushing occurs only once when the lock is released. This results in less disk activity.

e

bl a r fe

s

9.3.1

Explicit Table Locks

an r t n

no a s a h 9.3.1.1 LOCK TABLES statement n) ideฺ v ฺ The LOCK TABLES statement names each table to be locked and the type of lock to be acquired. The m u o G c following statement acquires a read lock on the Country 1ฺ tableeandnta write lock on the City table: 2 s t pWRITE; mysql> LOCK TABLES Country READ, City tud s S @ this n a The following are a list of issues that be addressed when using the LOCK TABLES statement: r t must e ฺ s n • All locks must u- If any of the tables to be locked are already in use, LOCK TABLES obe acquired o o t does not return until it has acquired all of the requested locks. m ( nse n a • r Locking multiple tables - If multiple tables need to be used while holding an explicit lock, all of ceto be locked T i l them need at the same time because no unlocked tables can be used while the explicit t e y lock is held. Also, all the tables must be locked with a single LOCK TABLES statement. LOCK u g TABLES releases any locks that are already held, so it cannot be issued multiple times to acquire N 256

Clients manage explicit table locks with two statements. LOCK TABLES acquires table locks and UNLOCK TABLES releases them.

multiple locks.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-4

MySQL Developer Techniques 257

LOCK TABLES types The following list describes the available LOCK TABLES types and their effects: •

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 9:Locking Levels

•

•

READ - Locks a table for reading. o A READ lock locks a table for read queries such as SELECT that retrieve data from the table. It does not allow write operations such as INSERT, DELETE, or UPDATE that modify the table, even by the client that holds the lock. When a table is locked for reading, other clients can read from the table at the same time, but no client can write to it. A client that wants to write to a table that is read-locked must wait until all clients currently reading from it have finished and released their locks. WRITE - Locks a table for writing. o A WRITE lock is an exclusive lock. It can be acquired only when a table is not being used. Once acquired, only the client holding the write lock can read from or write to the table. Other clients can neither read from nor write to it. No other client can lock the table for either reading or writing. READ LOCAL - Locks a table for reading, but allows concurrent inserts. o A concurrent insert is an exception to the “readers block writers” principle. It applies only to MyISAM tables. If a MyISAM table has no holes in the middle resulting from deleted or updated records, inserts always take place at the end of the table. In that case, a client that is reading from a table can lock it with a READ LOCAL lock to allow other clients to insert into the table while the client holding the read lock reads from it. If a MyISAM table does have holes, they can be removed by using OPTIMIZE TABLE to defragment the table. o A READ LOCAL lock can be acquired for a fragmented MyISAM table, or for a non-MyISAM table, but in such cases, concurrent inserts are not allowed. The lock acts like a regular READ lock. LOW_PRIORITY WRITE - Locks a table for writing, but acquires the lock with a lower priority. o With this lock type, if the client must wait for the lock, other clients that request read locks during the wait are allowed to get their locks first. A normal write lock request is satisfied when no other clients are using the table. If other clients are using the table when the request is made, it waits until those clients have finished. A LOW_PRIORITY WRITE lock request also waits for any new read requests that arrive while the lock request is pending.

e

bl a r fe

s

an r t n

T

no a s a h n) ideฺ v ฺ om t Gu c ฺ 1 n 2 e s d t • sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r TABLE ce statements T 9.3.1.2 tUNLOCK i l release explicit locks, the client holding the lock must issue an UNLOCK TABLES statement. This uyeTo 258 g statement names no tables, because it releases all explicit locks held by the issuing client. Other ways that hi N

explicit locks are released: • LOCK TABLES reissued - Explicit locks held by a client also are released if the client issues another LOCK TABLES statement. • Client termination - Locks cannot be maintained across connections; if a client has any unreleased locks when its connection to the server terminates, the server implicitly releases its locks. • Administrator KILL - An administrator with the SUPER privilege can terminate a client connection with the KILL statement, which causes release of locks held by the client. • START TRANSACTION - Table locks may be affected by transactions and vice versa. Beginning a transaction with START TRANSACTION causes an implicit UNLOCK TABLES. Issuing a LOCK TABLES statement will implicitly commit any pending transaction. If any tables are already locked, issuing an UNLOCK TABLES statement will implicitly commit any pending transaction. _________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-5

MySQL Developer Techniques

9.3.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

259

Thi

Chapter 9:Locking Levels

Explicit Row Locks MySQL (specifically the InnoDB storage engine) implements standard row-level locking utilizing two types of locks: • A shared lock allows a transaction to read a row. • An exclusive lock allows a transaction to update or delete a row. Shared row locks Performing a read in share mode means that latest available data is read, and then a shared mode lock is set on the rows that were read. This is implemented by using the following SELECT syntax:

mysql> SELECT CountryCode FROM City WHERE Code LIKE 'A%' LOCK IN SHARE MODE;

260

A shared mode lock prevents others from updating or deleting the rows that have been read. Also, if the latest data belongs to a yet uncommitted transaction of another client connection, the server will wait until that transaction commits. Exclusive Row Locks Performing a read in exclusive mode means that the latest available data is read, and then an exclusive lock is set on the rows that were read. This is implemented by using the following SELECT syntax:

e

bl a r fe

s

an r t n

no a s or deleting the rows Similar to shared mode locks, an exclusive mode lock prevents others fromaupdating h that have been read. However, the exclusive mode lock is different from shared mode locks in that other ) ฺ nthat e v users can not perform a lock (either shared or exclusive) on the rows have been read. ฺ m areGreleased uid when the transaction is o Locks set by IN SHARE MODE and FOR UPDATE reads c 1ฺ ent committed or rolled back. 2 s pt Stud s SELECT @... tFOR isUPDATE not working? n h a r Locking of rows for update using SELECT only applies when autocommit is disabled (either by ฺt u... seFORor byUPDATE n o beginning transaction with START TRANSACTION setting AUTOCOMMIT to 0). If autocommit is enabled, the o o t m rows matching the specification are not locked. n ( ense a r T lic t e uy g N mysql> SELECT CountryCode FROM City WHERE Code LIKE 'A%' FOR UPDATE;

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-6

MySQL Developer Techniques

9.4

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

261

9.4.1

Chapter 9:Locking Levels

Storage Engine Locking Techniques The algorithms used by MySQL concerning locking methods are completely dependent upon the storage engines that are being utilized. Each storage engine is very specific in its implementation of locks. The two most popular storage engines supported by MySQL, MyISAM and InnoDB, will be discussed in detail.

MyISAM Locking The MyISAM storage engine protects against corruption of data by different threads updating in conflicting ways by using table-level locking. The MyISAM storage engine utilizes one of three types of locks depending on the request issued by the connecting thread: •

READ LOCAL (SHARED)– This type of lock allows an INSERT to append only to the end of the data file. If the INSERT requires a push of data anywhere else but to the end of the data file (beginning or middle of the data file), the INSERT statement is delayed until the READ LOCAL LOCK is released. This is the default lock type for SELECT statements.

e

T

bl a r • READ (SHARED) –When this lock is activated, all UPDATE, INSERT and DELETE fe s n statements are delayed until the lock is released. READ locks are shared locks. tra n • WRITE (EXCLUSIVE)– This type of lock is activated whenever an UPDATE no or DELETE a request is submitted, or an INSERT is received that would require adding the data anywhere but s until the lock is released. a to the end of the data file. This lock prevents any activity on the table h WRITE locks are exclusive locks. n) ideฺ v ฺ om t Gu Tables designed toฺc Log Activity n 21INSERT e s d With the MyISAM storage engine's capability to perform operations SELECT queries are running t p Stanu excellent choicewhile against the data (through the READ LOCAL lock s type), it makes for applications that are responsible @end tusers isthe ability to run reports or other query operations on the data n for logging continual activities while also giving h a r eMyISAM knows that all new records will always occur at the end of contained. This is made possiblen byฺtthe fact that s u o the data file and thus there iso no need to hold to up SELECT statements that request data from anywhere else in the table. m ( e ns an Inserts Concurrent r e c T li t 262 e A concurrent insert is an INSERT that happens at the same time as SELECT's on the same table. With y gu the MyISAM storage engine, it is possible to force SELECT statements to use the READ LOCAL lock N regardless if there is room in the middle of the data file for the INSERT to fill. Whether an INSERT will hi be a concurrent insert is determined by the concurrent_insert global status variable. This variable can be assigned by setting the mysqld --concurrent_insert or SET GLOBAL concurrent_insert parameter. The concurrent_insert parameter can take any one of the following values: •

0 - (Off) This tells the server to not allow any concurrent inserts.

•

1 - (Default) Enables concurrent insert for MyISAM tables that don't have holes.

•

2 - Enables concurrent inserts for all MyISAM tables. If the table has a holes and is in use by another thread the new row will be inserted at the end of the table. If the table is not in use, MySQL obtains a normal WRITE lock and inserts the new row into the holes.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-7

MySQL Developer Techniques

Priorities When running a SQL statement that interacts with a MyISAM table, an individual user can determine the importance level that their individual statements have in regards to priority execution on the MySQL server. There are four levels of importance in relation to database actions: 1. Highest priority - The database actions that this level of priority (or importance) include SELECT or INSERT commands that the user or application interfacing with the server has identified as needing to run before any other SQL statements waiting to be executed by the server.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

263

Thi

Chapter 9:Locking Levels

2.

Normal writes - If there are no high priority SQL statements waiting to be executed, the MySQL server will allow INSERT, DELETE and UPDATE commands to run first prior to any SELECT statements that are waiting to be executed by the server.

3.

Normal reads - As stated, SELECT statements are at the bottom of the execution list (unless identified as high priority) when waiting for execution by the server.

4.

Lowest priority - The database actions at this level of priority will wait for all other SQL statements to execute prior to actually executing themselves. The end user of the application interfacing with the server must specifically tell the server that these SQL statements are considered low priority otherwise the normal writes and reads order will be implemented.

e

bl a r fe

s

The following is a list of the MySQL priority commands that can be used:

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e •

an r t n

HIGH_PRIORITY – This command gives the SQL statement being executed the ability to move to the top of the list of any waiting SQL statements (from all threads) and can be used with the following SQL statements: o SELECT - When SELECT HIGH_PRIORITY ... is utilized, MySQL gives the SELECT a higher priority than a statement that updates a table. This should only be used for queries that are very fast and must be done at once. A SELECT HIGH_PRIORITY query that is issued while the table is locked for reading runs even if there is an update statement waiting for the table to be free. HIGH_PRIORITY cannot be used with SELECT statements that are part of a UNION. o INSERT – When INSERT HIGH_PRIORITY ... is utilized, MySQL stops any concurrent insert capabilities and gives the INSERT statement complete control of the table until the statement has executed completely.

uy g N

Priority levels When using the INSERT HIGH_PRIORITY statement, the server automatically overrides any existing low_priority_updates setting. The low_priority_updates setting, when set to 1, forces all INSERT, UPDATE, DELETE, and LOCK TABLE WRITE statements to wait until there is no pending SELECT or LOCK TABLE READ on the affected table. This variable previously was named sql_low_priority_updates. In addition, by default writing queries have a higher priority than reading queries.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-8

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

•

Thi

Chapter 9:Locking Levels

LOW_PRIORITY – This command places the SQL statement being executed at the bottom of the list of any waiting and new SQL statements (from all threads) and will only run when all waiting and new SQL statements have completed. This command can be used with the following SQL statements: o INSERT – When INSERT LOW_PRIORITY ... is utilized, MySQL delays the execution of the statement (and stops any other work from the particular client) until no other clients are reading from the table. This includes any other clients that may access the table while the INSERT LOW_PRIORITY statement is waiting to execute. In an environment where there is a large number of clients that can be reading from the server at any one time this could cause the client to wait a very long time. This should normally not be used with MyISAM tables because doing so disables concurrent inserts. o UPDATE – The UPDATE LOW_PRIORITY ... works identical to the INSERT LOW_PRIORITY statement and should also be avoided unless absolutely necessary. o DELETE – The DELETE LOW_PRIORITY ... also works the same as the INSERT LOW_PRIORITY by not executing until all clients are done reading from the table. This can cause integrity issues in that another client can be viewing data that should be deleted immediately.

e

bl a r fe

s

Lock starvation

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Using LOW_PRIORITY with UPDATES, INSERTS and DELETES can cause the queries to wait in the queue forever, if there is no time of day when clients aren't reading from the table.

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-9

MySQL Developer Techniques

9.4.2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

264

Chapter 9:Locking Levels

InnoDB Locking The InnoDB storage engine supports row-level locking, providing the finest level of lock granularity, where only the row that is read or updated is locked; and table-level locking, which by default is only used when there are changes to the table structure itself (as is the case with ALTER TABLE). • Row-level locking - With row-level locking, InnoDB allows other concurrent transactions to access other rows in the table. These locks are set using the primary key index structure. Any query that utilizes locks and does not have restrictions on any index will lock all examined rows.

Next-Key Locking In row level locking InnoDB uses next-key locking. This means that besides index records, InnoDB can also lock the 'gaps' before or after an index record to block insertions by other users immediately before the index record. A next-key lock means a lock which locks an index record and the gaps before it. A gap lock means a lock which only locks a gap before some index record. •

e

Table-level locking - InnoDB uses table locks implicity all the time. The following describes the different ways that this works: o

bl a r fe

s

AUTO_INCREMENT columns - When an InnoDB table contains an AUTO_INCREMENT column, a table level lock is consistently used. The purpose of this is to guarantee the uniqueness of auto-inc values, and make them properly sequential with no gaps. In MySQL 5.1, there are ways to override this feature; however, there remain situations where auto-inc statements require table level locks to avoid problems with concurrency.

an r t n

T

no a s a h n) ideฺ v ฺ o Intention locks - InnoDB sets "intention"m locks on tables that it intends to place row uto the oremain G level locks on. Intention locks usually invisible application, but they are c ฺ t 1 andemaking n critical to things like deadlocksprevention MySQL's explicit table level locks 2 work. pt Stud s @ (--transaction-isolation) TRANSACTION ISOLATION n LEVEL is h a t r 265 t storagesengine e that affects locking is the transaction locking level that is Another aspect of the InnoDB u onฺ levels used. Transaction o isolation specify what data is visible to statements within a transaction. These to m levels directly(impact the e level of concurrent access by defining what interaction is possible between nssame target data source. InnoDB offers all four transaction isolation levels transactions the an against r e c T described by theliSQL standard: t e y • READ UNCOMMITTED (weak protection) - SELECT statements are performed in a nongu N locking fashion, but a possible earlier version of a record might be used. Thus, using this hi

isolation level, such reads are not consistent. The bottom line problem is that open transactions see changes seen by other open transactions, and if the other open transactions rollback, the transactions end up using data that was never committed. Thus in essence these transactions never really "existed". This is also called a “dirty read.” This setting is rarely used as a default setting. READ UNCOMMITTED is the lowest transaction isolation level and can lead to many problems with the data if used in a production environment with multiple users. However, there are times when a developer would want to use this transaction isolation level: o

Changes not finished - The database developer/administrator may want to view some work that is on-going, such as a batch job that is currently running. By using READ UNCOMMITTED for a specific session, the database administrator could see changes that are not yet finished.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-10

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Chapter 9:Locking Levels

o

Reducing InnoDB locking overhead - When a developer knows their application's INSERT/UPDATE patterns are very specific and will not result in dirty reads, then READ UNCOMMITTED is fine; this setting provides greater concurrency because it reduces InnoDB locking overhead.

o

SELECT's only - When the developer does not care about dirty reads for some of their clients. Clients that simply read data, and have no need to update it, are prime candidates for considering READ UNCOMMITTED at the session level.

•

READ COMMITTED (better protection) - All SELECT ... FOR UPDATE and SELECT ... LOCK IN SHARE MODE statements lock only the index records, not the gaps before them, and thus allow the free insertion of new records next to locked records. UPDATE and DELETE statements using a unique index with a unique search condition lock only the index record found, not the gaps before it. With this level selected, the changes made by all COMMITTED transactions are made as soon as they are committed.

•

REPEATABLE READ (good protection) - This is the default isolation level of InnoDB. SELECT ... FOR UPDATE, SELECT ... LOCK IN SHARE MODE, UPDATE, and DELETE statements that use a unique index with a unique search condition lock only the index record found, not the gaps before it. With other search conditions, these operations employ nextkey locking, locking the index range scanned with next-key or gap locks, and block new insertions by other users. With this level selected, data reads within the same transaction always see tables in the same consistent state even if there are transactions committed between the reads.

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ • SERIALIZABLE (zero anomalies) - This levelm like REPEATABLE READ, but InnoDB u oto isSELECT G c implicitly converts all plain SELECT statements ... LOCK IN SHARE MODE. ฺ t 1 n 2significant e This level is rarely used as it can incur performance penalties. The point with the s d t p transactions tu could be replayed serially and it would name is that in this mode two s interlacing S give the same result. @ s i n h a t r t e onฺ to usNon-locking SELECT's o In each of the InnoDB transaction levels (except SERIALIZABLE), normal SELECT statements are non-locking. (m nsisolation ecompleted n Non-locking SELECT statements are InnoDB through the use of Multiversion Concurrency Control (MVCC). a r ce for improving indatabase i MVCC is tanT advanced technique performance in a multiuser environment. The InnoDB tablespace l e y contains many versions of the same rows in order to maintain between transactions. Different transactions see u versions of the same rows. Ultimately, each SELECT isolation g different statement works from a snapshot of the data taken at the hi Ntime of reading.

T

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-11

MySQL Developer Techniques

Chapter 9:Locking Levels

9.4.2.1 Deadlocks in InnoDB Deadlocks are a specific condition when two or more processes are each waiting for another to release a resource, or more than two processes are waiting for resources in a circular chain. InnoDB automatically detects a deadlock of transactions and rolls back a transaction or transactions to break the deadlock. InnoDB tries to pick small transactions to roll back, where the size of a transaction is determined by the number of rows inserted, updated, or deleted.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

266

System variable settings - InnoDB is aware of table locks only if innodb_table_locks=1 (the default) and AUTOCOMMIT=0, and the MySQL layer above it knows about row-level locks. Otherwise, InnoDB cannot detect deadlocks where a table lock set by a MySQL LOCK TABLES statement or a lock set by a storage engine other than InnoDB is involved. These situations must be resolved by setting the value of the innodb_lock_wait_timeout system variable. • Rollback releases locks, sometimes - When InnoDB performs a complete rollback of a transaction, all locks set by the transaction are released. However, if just a single SQL statement is rolled back as a result of an error, some of the locks set by the statement may be preserved. This happens because InnoDB stores row locks in a format such that it cannot know afterward which lock was set by which statement. Avoiding Deadlocks There are some steps that database developers can follow to minimize the possibility of deadlocking. •

e

bl a r fe

s

267

no a s • Commit often - Small transactions are less prone to collision. a h • Lowest isolation level possible - Having an isolation level is greater then what is needed n)that eฺpossible v d can add greater complexity to the transaction process. ฺUsing the lowest level, i m Guthe possibility isolation o while still ensuring that the data is safe, can help minimize of a deadlock c 1ฺ ent condition. 2 s udtransactions are set up to access tables and • Access tables and rows in a fixed - When ptorder S t s rows in a fixed order, the possibility ofis deadlocks occurring is greatly reduced. This is due to nor@row being h a t the fact that each ttable locked in the same order will ensure that no other r e ฺ s transaction running can goucounter to the order thus ensuring a one way direction for updates n o tupdates o o A and then B, whereas another transaction updates B and then A, (e.g.. If a transaction m ( e the system would be prone to deadlocks). s n n a r e Hope worst Tfor the best, licprepare forof the t e Even with an understanding deadlocks and ways to prevent them from occuring, a developer must uy

g

N Thi

an r t n

prepare their applications to handle deadlock conditions. Due to the fact that a deadlock condition will rollback a transaction, the application must be designed to recognize that a rollback occurred and reissue the transaction automatically to prevent data loss or lack of data integrity.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-12

MySQL Developer Techniques

9.4.3

Locking with Other Storage Engines MyISAM and the InnoDB storage engine are currently the most popular and robust storage engines available for the MySQL server. However, there are other storage engines that address locking also and are listed below:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

268

Thi

Chapter 9:Locking Levels

•

MEMORY - The MEMORY storage engine utilizes table-level locking and acts similar to the MyISAM storage engine in its implementation.

•

Falcon - The Falcon storage engine uses MVCC (multi-version concurrency control) to provide record-level concurrency. MVCC is designed to enable records and tables to be updated without the overhead associated with row-level locking mechanisms. The MVCC implementation virtually eliminates the need to lock tables or rows during the update process.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-13

MySQL Developer Techniques

9.5

Locking Issues A few locking issues that arise with applications using MySQL are:

269

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 9:Locking Levels

•

Blocked operation – This takes places when an operation is prevented from completing due to a lock on the data in question.

•

Hold lock - The execution time of statements that create a lock on the data accessed must be considered to ensure that the performance of other statements are not degraded.

•

Deadlocks - This is a specific condition when two or more processes are each waiting for another to release a resource, or more than two processes are waiting for resources in a circular chain. This type of condition can not occur on storage engines that only support table level locking.

•

Lock starvation - This condition is similar to the deadlock issue discussed, but with lock starvation a transaction is prevented from proceeding for an indefinite amount of time while other transactions continue to run as expected. Lock starvation is the result of other transactions that are running fine hogging the server thus preventing the transaction in question being "starved" from server resources.

e

bl a r fe

ns a r t • Lock timeouts - Within MySQL, there is a setting called table_lock_wait_timeout that n o determines how long a lock a statement will wait to acquire a lock before timing out and n returning an error. The default setting for this variable is 50 seconds. a s a h Many of the issues can be minimized or completely eliminated by using smaller ฺ n) idetransactions. v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t uye

g

N Thi

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-14

MySQL Developer Techniques

Chapter 9:Locking Levels

Inline Lab 9-A

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this lab you will initiate queries for testing and reviewing of the way the MyISAM and InnoDB storage engines handle implicit locking.

Thi

Step 1: Verify that you are using MyISAM tables In the world database, type the following MySQL SHOW statement to determine if the are using the MyISAM storage engine: mysql> SHOW TABLE STATUS\G +-----------------+--------+---------+------------+------+---| Name | Engine | Version | Row_format | Rows | ... +-----------------+--------+---------+------------+------+---| City | MyISAM | 10 | Fixed | 4079 | | country | MyISAM | 10 | Fixed | 239 | | countrylanguage | MyISAM | 10 | Fixed | 984 | +-----------------+--------+---------+------------+------+---3 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h ฺ MyISAM storage nis)not using ethe v ฺ We will be using the City table in this lab. If the City table d i omengine engine, use the following SQL statement to change the c storage to u MyISAM: G ฺ t 1 n 2 ts tude mysql> ALTER TABLE City ENGINE=MyISAM; p s S Query OK, 4079 rows affected (#.## sec) @ Records: 4079 Duplicates: 0 n Warnings: 0is h tra se t ฺ n o to u o Step 2: Create an SQL m ( nsstatement e that will lock the City table n a rsame terminal In the cewindow from step 1, enter the following SQL statement which will produce a T i l t every lengthy running query: guy The SHOW TABLE STATUS command will produce a large amount of information concerning the tables in the database. In this case, the tables listed include all those tables in the world database (yours may look different based on the status of your world database).

N

mysql> SELECT c1.Name FROM City AS c1 JOIN City AS c2 -> JOIN City AS c3 ORDER BY RAND() LIMIT 1;

This query will take some time to run and will give you a chance to test the implicit locking capability of the MyISAM storage engine.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-15

MySQL Developer Techniques

Chapter 9:Locking Levels

Step 3: Test SELECT, INSERT and UPDATE statements against the City table

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Now that the SQL statement from step 2 is continuing to run, open up a second terminal window and follow the steps from step 1 to log in and use the world database.

Thi

Once you have a second mysql client running and are using the world database, type the following SQL statement to see if you can retrieve data from the City table: mysql> SELECT Name FROM City LIMIT 1; +-------+ | Name | +-------+ | Kabul | +-------+ 1 row in set (#.## sec)

From this response, it can be determined that the City table is not locked in reference to other clients being able to retrieve data from it. Now lets try to perform an INSERT against the City table while it is being read by another client. Type the following SQL statement to see if you can add a record to the City table:

e

bl a r fe

s

an r t n

no a mysql> INSERT INTO City VALUES (NULL,'Sakila','AUS','',1); as h ฺ Query OK, 1 row affected (#.## sec) ) n ฺv uide m o Due to the fact that the record could be added to the end of the City (versus somewhere in the Gbetableadded. ฺc the record t 1 n middle of the data), the MyISAM storage engine2 allowed to However, let's try an e s d t update by entering the following SQL statement: sp s Stu @ iName='Sakila'; n WHERE mysql> UPDATE City SET Name='Test' h a t r t e s onฺ to auresponse This SQL statementowill not return (and will freeze the activity of this client) due to the fact m ( e that the SQL statement from Step 2 is still executing. update can not take place until that SQL sthe fact that the record wouldThe nfinishes due n a statement to need to alter records that are involved in the r e c T i l t query from step 2. e guy

N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-16

MySQL Developer Techniques

Chapter 9:Locking Levels

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Step 4: View the City table status

Thi

Due to the fact that the first mysql client is running the SQL SELECT statement from step 2 and the second mysql client is waiting to execute the UPDATE statement from step 3, it will be necessary to open up a third mysql client in a third terminal window and follow the steps from Step 1 to log in to the MySQL server. Once you have a third mysql client running, type the following SQL statement to view the status of the running processes on the server: mysql> SHOW PROCESSLIST\G *************************** 1. row *************************** Id: 1 User: root Host: localhost db: world Command: Query Time: 22 State: Copying to tmp table on disk Info: SELECT c1.Name FROM City AS c1 JOIN City AS c2 JOIN City AS c3 JOIN City AS c4 JOIN City AS c5 ORDER *************************** 2. row *************************** Id: 2 User: root Host: localhost db: world Command: Query Time: 7 State: Locked Info: UPDATE City SET Name='Test' WHERE Name='Kabul' *************************** 3. row *************************** Id: 3 User: root Host: localhost db: NULL Command: Query Time: 0 State: NULL Info: SHOW PROCESSLIST 3 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

The status of the first client (with an id of 1) is showing that it is still executing and creating a temporary table to handle the query. The status of the second client (with an id of 2) is showing that it is locked due to the first client still running. As far as the third client, that is our current connection and the NULL in the State column identifies that the query has completed successfully.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-17

MySQL Developer Techniques

Chapter 9:Locking Levels

Step 5: Terminate the queries still executing

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In the current mysql client (the third client), type the following commands (in order to prevent the update from actually executing):

Thi

mysql> KILL QUERY 2; Query OK, 0 rows affected (#.## sec)

This will kill the query the UPDATE query that is running in the mysql client that is identified by id 2 (you may have to replace the client id with the id that you were given in Step 4). Using KILL QUERY instead of KILL or KILL CONNECTION allows for the client to continue running after the query it was executing is terminated. Type the following command to terminatate the SELECT query that is running in the mysql client that is identified by id 1 (you may have to replace the client id with the id that you were given in Step 4): mysql> KILL QUERY 1; Query OK, 0 rows affected (#.## sec)

e

bl a r fe

s

an r t n

no a s a h In the first mysql client, type the following command to change the n)storageidengine eฺ of the City table v ฺ to the InnoDB storage engine: om t Gu c ฺ mysql> ALTER TABLE City ENGINE=InnoDB; 21 den s Query OK, 4079 rows affected (#.## sec) t sp 0s Stu Records: 4079 Duplicates: 0 Warnings: @ i n h a t r t e ฺ the ability This will provide you with the same steps that were used against the City table sto runstorage u onusing when the City table was the MyISAM engine. o o t m ( e n eSQL nsstatement that will lock the City table StepT7:ra Execute the c li t e y gu In the same terminal window from step 6, enter the SQL statement that was run against the City table Step 6: Alter the City table to use the InnoDB storage engine

N

in step 2:

mysql> SELECT c1.Name FROM City AS c1 JOIN City AS c2 -> JOIN City AS c3 ORDER BY RAND() LIMIT 1;

This query will take some time to run and will give you a chance to test how the InnoDB storage engine deals with the possibity of implicit locking.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-18

MySQL Developer Techniques

Chapter 9:Locking Levels

Step 8: Test SELECT, INSERT and UPDATE statements against the City table (using InnoDB)

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

With the SQL statement from step 7 continuing to run, go back to the second terminal window (that was opened in step 3) and type the following SQL statement to see if you can retrieve data from the City table:

Thi

mysql> SELECT Name FROM City LIMIT 1; +-------+ | Name | +-------+ | Kabul | +-------+ 1 row in set (#.## sec)

From this response, it can be determined that the City table is not locked in reference to other clients being able to retrieve data from it. Now lets try to perform an INSERT against the City table while it is being read by another client. Type the following SQL statement to see if you can add a record to the City table:

e

bl a r fe

s

an r t n

no a s a h ) allowsetheฺ record to be added. Similar to the MyISAM storage engine, the InnoDB storage enginen also v ฺ id running SQL statement; Both a SELECT and INSERT worked while the table was tied up with au long m o G c however, will the InnoDB storage engine allow an update to take place in the data that is currently ฺ t 1 n being read: 2 e pts Stud s mysql> UPDATE City SET Name='Test' WHERE is Name='Sakila'; @ n Query OK, 2 rows affected (#.## sec) th a r ฺt2 Warnings: Rows matched: 2 Changed: se 0 n u o o to m ( e Unlike thenUPDATE of s the City table (when using the MyISAM storage engine) in step 3, this nthrough aactuallyicgoes r e UPDATE while the SELECT statement is running from step 7. This is due to the T l storage engine t fact that the InnoDB utilizes row-level locking and thus will allow data to be altered e guy while there are searches going on against any other row except the current row that is locked. mysql> INSERT INTO City VALUES (NULL,'Sakila','AUS','',1); Query OK, 1 row affected (#.## sec)

N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-19

MySQL Developer Techniques

9.6

Summary In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

270

Thi

Chapter 9:Locking Levels

•

Explain locking fundamentals

•

Utilize explicit table locks

•

Explain the different locking techniques of the storage engines

•

Explain how InnoDB's row-locking and next-key-locking work

•

Describe the most common issues associated with locking

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

9-20

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 10

e

s

bl a r fe

an r t n

no CREATING REPORTS a as

uy g N

h ฺ ) n ฺv uide m o ฺc nt G 1 2 e pts Stud s @ this n a r ฺt use n o o to m ( e an icens r T l et

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

Chapter 10:Creating Reports

10. CREATING REPORTS

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

10.1.

Thi

Overview This chapter provides the knowledge and skills to build advanced queries used for reporting management information to end users. At the completion of this chapter, you will be able to:

272

•

Use functions to calculate multiple conditions

•

Create calculated reports

•

Create quarterly reports

•

Create cross tab reports

•

Create a bar chart with SQL

•

Create a decision table to avoid hard wiring of logic in the application code

e

s

Create materialized views

an r t n

o

an s a h ) • Produce sequential or missing data n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e •

bl a r fe

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-1

MySQL Developer Techniques

Chapter 10:Creating Reports

10.2. Calculate Multiple Conditions

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

273

Aggregate functions in SQL is one of the major features and assets of database systems. For the most part, calculating aggregate data is straightforward and simple enough to create detailed data with little to no creative SQL having to be implemented. Data such as how many countries are represented in the dataset is as simple as performing the following aggregate SQL statement: mysql> SELECT COUNT(Code) FROM Country; +-------------+ | COUNT(Code) | +-------------+ | 239 | +-------------+ 1 row in set (#.## sec)

e

T

bl This can be useful for some situations, but for most situations a little bit more detail about the data isera sf needed. n a -tr Conditional counting n 274 o n that breaks Of course, taking this counting to the next step might result in creating an SQL statement a this down the countries based on the continent that they are located in. In MySQL much s aa conditionisinactually h simpler than it sounds. In most database systems, a developer can)place an aggregate n of theidcondition function but MySQL makes it even simpler by reducing the ฺlength eฺ that needs to be v entered. For example, the following SQL statement demonstrates how to conditions in aggregate Gofuthecreate comthatnsome ฺ functions that count more specifically based on the 1 continent countries belong to: t s2 tude t p mysql> SELECT SUM(Continent='Asia')sAS Asia,SSUM(Continent='Europe') AS Europe, @ this -> FROM Country; n a +------+--------+ r ฺt use | Asia | Europe | n o +------+--------+ o to m | 51 | 46(| e n ens +------+--------+ a r T 1 row tin set (#.## licsec) e uy g This ability to place conditions in an aggregate function eliminates the need to create a WHERE filter with hi N multiple subqueries to produce a similar output.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-2

MySQL Developer Techniques Taking it to the next level

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

275

Thi

Chapter 10:Creating Reports

The resulting data is getting much more useful, but there is still another level that this can be taken. By adding some creative SQL, the approximate percentages (based on all the data) can be derived: mysql> SELECT -> ROUND(100*SUM(Continent='Asia')/COUNT(Code),1) AS Asia, -> ROUND(100*SUM(Continent='Europe')/COUNT(Code),1) AS Europe -> FROM Country; +------+--------+ | Asia | Europe | +------+--------+ | 21.3 | 19.2 | +------+--------+ 1 row in set (#.## sec)

e

The ROUND(equation,1) produces a more accurate percentage by bringing the value out 1 decimal place. Multiple conditions can also be placed into the aggregate functions to get more specific.

276

bl a r fe

s

an r t n

mysql> SELECT -> ROUND(100*SUM(Continent='Asia' AND Population > 1000000)/COUNT(Code),1) -> AS Asia_Large, -> ROUND(100*SUM(Continent='Asia' AND Population < 1000000)/COUNT(Code),1) -> AS Asia_Small, -> ROUND(100*SUM(Continent='Europe' AND Population > 1000000)/COUNT(Code),1) -> AS Europe_Large, -> ROUND(100*SUM(Continent='Europe' AND Population < 1000000)/COUNT(Code),1) -> AS Europe_Small -> FROM Country; +------------+------------+--------------+--------------+ | Asia_Large | Asia_Small | Europe_Large | Europe_Small | +------------+------------+--------------+--------------+ | 18.4 | 2.9 | 14.6 | 4.6 | +------------+------------+--------------+--------------+ 1 row in set (#.## sec)

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-3

MySQL Developer Techniques

Chapter 10:Creating Reports

10.3. Create a Calculated Report MySQL, like other databases, has a built-in function that handles the process of creating subtotals and totals without having to be creative with SQL. This built-in function is related to aggregate functions and is the WITH ROLLUP clause. Using the Country table, the following SQL creates a calculated report for population numbers by the country, averages based on the countries located in a continent and then finally an average for all countries:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

277

Thi

278

mysql> SELECT Name, Region, ROUND(AVG(Population),0) FROM Country -> WHERE Continent='North America' -> GROUP BY Region, Population WITH ROLLUP; +----------------------------------+-----------------+--------------------------+ | Name | Region | ROUND(AVG(Population),0) | +----------------------------------+-----------------+--------------------------+ | Anguilla | Caribbean | 8000 | | Montserrat | Caribbean | 11000 | | Turks and Caicos Islands | Caribbean | 17000 | | Virgin Islands, British | Caribbean | 21000 | | Saint Kitts and Nevis | Caribbean | 38000 | | Antigua and Barbuda | Caribbean | 68000 | | Dominica | Caribbean | 71000 | | Virgin Islands, U.S. | Caribbean | 93000 | | Grenada | Caribbean | 94000 | | Aruba | Caribbean | 103000 | | Saint Vincent and the Grenadines | Caribbean | 114000 | | Saint Lucia | Caribbean | 154000 | | Netherlands Antilles | Caribbean | 217000 | | Barbados | Caribbean | 270000 | | Bahamas | Caribbean | 307000 | | Martinique | Caribbean | 395000 | | Guadeloupe | Caribbean | 456000 | | Trinidad and Tobago | Caribbean | 1295000 | | Jamaica | Caribbean | 2583000 | | Puerto Rico | Caribbean | 3869000 | | Haiti | Caribbean | 8222000 | | Dominican Republic | Caribbean | 8495000 | | Cuba | Caribbean | 11201000 | | Cuba | Caribbean | 1589167 | | Belize | Central America | 241000 | | Panama | Central America | 2856000 | | Costa Rica | Central America | 4023000 | | Nicaragua | Central America | 5074000 | | El Salvador | Central America | 6276000 | | Honduras | Central America | 6485000 | | Guatemala | Central America | 11385000 | | Mexico | Central America | 98881000 | | Mexico | Central America | 16902625 | | Saint Pierre and Miquelon | North America | 7000 | | Greenland | North America | 56000 | | Bermuda | North America | 65000 | | Canada | North America | 31147000 | | United States | North America | 278357000 | | United States | North America | 61926400 | | United States | | 13053865 | +----------------------------------+-----------------+--------------------------+ 40 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

The resulting output includes subtotals (The 2nd listing for Cuba, Mexico and United States are the subtotals for the region) along with a total average for all the countries in North America (the last row in the set). This of course is the simplest way to create a calculated report, but it can be difficult to decipher (especially with large data sets). _________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-4

MySQL Developer Techniques

Using UNION The UNION syntax in SQL is used to combine the result from multiple SELECT statements into a single result set. With this syntax, the following SQL statement creates a similar response to what was seen with the WITH ROLLUP option:

279

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

280

Thi

Chapter 10:Creating Reports

mysql> SELECT Name, Region, Population, Population FROM Country -> WHERE Continent='North America' -> UNION -> SELECT NULL, Region, NULL, ROUND(AVG(Population),0) FROM Country -> WHERE Continent='North America' -> GROUP BY Region ORDER BY Region; +----------------------------------+-----------------+------------+------------+ | Name | Region | Population | Population | +----------------------------------+-----------------+------------+------------+ | Grenada | Caribbean | 94000 | 94000 | | Puerto Rico | Caribbean | 3869000 | 3869000 | | Aruba | Caribbean | 103000 | 103000 | | Saint Vincent and the Grenadines | Caribbean | 114000 | 114000 | | Jamaica | Caribbean | 2583000 | 2583000 | | Turks and Caicos Islands | Caribbean | 17000 | 17000 | | Cayman Islands | Caribbean | 38000 | 38000 | | Martinique | Caribbean | 395000 | 395000 | | NULL | Caribbean | NULL | 1589167 | | Dominican Republic | Caribbean | 8495000 | 8495000 | | Anguilla | Caribbean | 8000 | 8000 | | Saint Kitts and Nevis | Caribbean | 38000 | 38000 | | Bahamas | Caribbean | 307000 | 307000 | | Haiti | Caribbean | 8222000 | 8222000 | | Antigua and Barbuda | Caribbean | 68000 | 68000 | | Guadeloupe | Caribbean | 456000 | 456000 | | Saint Lucia | Caribbean | 154000 | 154000 | | Barbados | Caribbean | 270000 | 270000 | | Trinidad and Tobago | Caribbean | 1295000 | 1295000 | | Virgin Islands, British | Caribbean | 21000 | 21000 | | Cuba | Caribbean | 11201000 | 11201000 | | Virgin Islands, U.S. | Caribbean | 93000 | 93000 | | Dominica | Caribbean | 71000 | 71000 | | Montserrat | Caribbean | 11000 | 11000 | | Netherlands Antilles | Caribbean | 217000 | 217000 | | Guatemala | Central America | 11385000 | 11385000 | | Belize | Central America | 241000 | 241000 | | Nicaragua | Central America | 5074000 | 5074000 | | Costa Rica | Central America | 4023000 | 4023000 | | Mexico | Central America | 98881000 | 98881000 | | NULL | Central America | NULL | 16902625 | | El Salvador | Central America | 6276000 | 6276000 | | Panama | Central America | 2856000 | 2856000 | | Honduras | Central America | 6485000 | 6485000 | | Greenland | North America | 56000 | 56000 | | Saint Pierre and Miquelon | North America | 7000 | 7000 | | Bermuda | North America | 65000 | 65000 | | Canada | North America | 31147000 | 31147000 | | United States | North America | 278357000 | 278357000 | | NULL | North America | NULL | 61926400 | +----------------------------------+-----------------+------------+------------+ 40 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

This SQL produces the subtotals that were found in the earlier SQL using the WITH ROLLUP clause, but this time it is clear which one is the subtotal row based on the NULL value in the Name and first Population column. _________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-5

MySQL Developer Techniques 281

Chapter 10:Creating Reports

Improving UNION

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Now of course, the location of the subtotal row is in the middle of most of the groups versus at the bottom where they should be. This can be corrected by using the COALESCE function with a number greater than any of the populations to move the NULL field to the bottom of the groups as shown below:

Thi

282

mysql> SELECT Name, Region, Population, Population FROM Country -> WHERE Continent='North America' -> UNION -> SELECT NULL, Region, NULL, ROUND(AVG(Population),0) FROM Country -> WHERE Continent='North America' -> GROUP BY Region ORDER BY Region, COALESCE(Population,6000000000); +----------------------------------+-----------------+------------+------------+ | Name | Region | Population | Population | +----------------------------------+-----------------+------------+------------+ | Anguilla | Caribbean | 8000 | 8000 | | Montserrat | Caribbean | 11000 | 11000 | | Turks and Caicos Islands | Caribbean | 17000 | 17000 | | Virgin Islands, British | Caribbean | 21000 | 21000 | | Cayman Islands | Caribbean | 38000 | 38000 | | Saint Kitts and Nevis | Caribbean | 38000 | 38000 | | Antigua and Barbuda | Caribbean | 68000 | 68000 | | Dominica | Caribbean | 71000 | 71000 | | Virgin Islands, U.S. | Caribbean | 93000 | 93000 | | Grenada | Caribbean | 94000 | 94000 | | Aruba | Caribbean | 103000 | 103000 | | Saint Vincent and the Grenadines | Caribbean | 114000 | 114000 | | Saint Lucia | Caribbean | 154000 | 154000 | | Netherlands Antilles | Caribbean | 217000 | 217000 | | Barbados | Caribbean | 270000 | 270000 | | Bahamas | Caribbean | 307000 | 307000 | | Martinique | Caribbean | 395000 | 395000 | | Guadeloupe | Caribbean | 456000 | 456000 | | Trinidad and Tobago | Caribbean | 1295000 | 1295000 | | Jamaica | Caribbean | 2583000 | 2583000 | | Puerto Rico | Caribbean | 3869000 | 3869000 | | Haiti | Caribbean | 8222000 | 8222000 | | Dominican Republic | Caribbean | 8495000 | 8495000 | | Cuba | Caribbean | 11201000 | 11201000 | | NULL | Caribbean | NULL | 1589167 | | Belize | Central America | 241000 | 241000 | | Panama | Central America | 2856000 | 2856000 | | Costa Rica | Central America | 4023000 | 4023000 | | Nicaragua | Central America | 5074000 | 5074000 | | El Salvador | Central America | 6276000 | 6276000 | | Honduras | Central America | 6485000 | 6485000 | | Guatemala | Central America | 11385000 | 11385000 | | Mexico | Central America | 98881000 | 98881000 | | NULL | Central America | NULL | 16902625 | | Saint Pierre and Miquelon | North America | 7000 | 7000 | | Greenland | North America | 56000 | 56000 | | Bermuda | North America | 65000 | 65000 | | Canada | North America | 31147000 | 31147000 | | United States | North America | 278357000 | 278357000 | | NULL | North America | NULL | 61926400 | +----------------------------------+-----------------+------------+------------+ 40 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-6

MySQL Developer Techniques 283

Chapter 10:Creating Reports

What about the final total

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Even though it may be slightly easier to read, the fact that there is no final total (like with the WITH ROLLUP clause) may prevent using this method over the built-in function. However, with a little bit more creative SQL, that can easily be corrected.

Thi

284

mysql> SELECT Name, Region, Population, Population FROM Country -> WHERE Continent='North America' -> UNION -> SELECT NULL, Region, NULL, ROUND(AVG(Population),0) FROM Country -> WHERE Continent='North America' -> GROUP BY Region -> UNION -> SELECT NULL, 'AVG Total', NULL, ROUND(AVG(Population),0) FROM Country -> WHERE Continent='North America' -> ORDER BY Region, COALESCE(Population,6000000000); +----------------------------------+-----------------+------------+------------+ | Name | Region | Population | Population | +----------------------------------+-----------------+------------+------------+ | Anguilla | Caribbean | 8000 | 8000 | | Montserrat | Caribbean | 11000 | 11000 | | Turks and Caicos Islands | Caribbean | 17000 | 17000 | | Virgin Islands, British | Caribbean | 21000 | 21000 | | Cayman Islands | Caribbean | 38000 | 38000 | | Saint Kitts and Nevis | Caribbean | 38000 | 38000 | | Antigua and Barbuda | Caribbean | 68000 | 68000 | | Dominica | Caribbean | 71000 | 71000 | | Virgin Islands, U.S. | Caribbean | 93000 | 93000 | ... | Jamaica | Caribbean | 2583000 | 2583000 | | Puerto Rico | Caribbean | 3869000 | 3869000 | | Haiti | Caribbean | 8222000 | 8222000 | | Dominican Republic | Caribbean | 8495000 | 8495000 | | Cuba | Caribbean | 11201000 | 11201000 | | NULL | Caribbean | NULL | 1589167 | | Belize | Central America | 241000 | 241000 | | Panama | Central America | 2856000 | 2856000 | | Costa Rica | Central America | 4023000 | 4023000 | | Nicaragua | Central America | 5074000 | 5074000 | | El Salvador | Central America | 6276000 | 6276000 | | Honduras | Central America | 6485000 | 6485000 | | Guatemala | Central America | 11385000 | 11385000 | | Mexico | Central America | 98881000 | 98881000 | | NULL | Central America | NULL | 16902625 | | Saint Pierre and Miquelon | North America | 7000 | 7000 | | Greenland | North America | 56000 | 56000 | | Bermuda | North America | 65000 | 65000 | | Canada | North America | 31147000 | 31147000 | | United States | North America | 278357000 | 278357000 | | NULL | North America | NULL | 61926400 | | NULL | AVG Total | NULL | 13053865 | +----------------------------------+-----------------+------------+------------+ 41 rows in set (0.00 sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

With the addition of another UNION, the total row is created and the result produced is similar to the WITH ROLLUP clause but slightly easier to read.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-7

MySQL Developer Techniques 285

Chapter 10:Creating Reports

Avoiding all confusion

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

With a little bit more creative SQL, this calculated report idea can be taken to the next level in readability. By adding two additional columns for subtotals and the grand total, the readability of the report can be improved.

Thi

286

mysql> SELECT Name, Region, Population, Subtotal, Total FROM ( -> SELECT Name, Region, Population, '' AS Subtotal, '' Total -> FROM Country WHERE Continent='North America' -> UNION -> SELECT NULL, Region, NULL, ROUND(AVG(Population),0), '' -> FROM Country WHERE Continent='North America' -> GROUP BY Region -> UNION -> SELECT 'AVG Total',NULL, NULL, '', ROUND(AVG(Population),0) -> FROM Country WHERE Continent='North America' -> ) AS Country_t ORDER BY Total, Region, COALESCE(Population,6000000000); +----------------------------------+-----------------+------------+----------+----------+ | Name | Region | Population | Subtotal | Total | +----------------------------------+-----------------+------------+----------+----------+ | Anguilla | Caribbean | 8000 | | | | Montserrat | Caribbean | 11000 | | | | Turks and Caicos Islands | Caribbean | 17000 | | | | Virgin Islands, British | Caribbean | 21000 | | | | Saint Kitts and Nevis | Caribbean | 38000 | | | | Cayman Islands | Caribbean | 38000 | | | | Antigua and Barbuda | Caribbean | 68000 | | | | Dominica | Caribbean | 71000 | | | | Virgin Islands, U.S. | Caribbean | 93000 | | | ... | Martinique | Caribbean | 395000 | | | | Guadeloupe | Caribbean | 456000 | | | | Trinidad and Tobago | Caribbean | 1295000 | | | | Jamaica | Caribbean | 2583000 | | | | Puerto Rico | Caribbean | 3869000 | | | | Haiti | Caribbean | 8222000 | | | | Dominican Republic | Caribbean | 8495000 | | | | Cuba | Caribbean | 11201000 | | | | NULL | Caribbean | NULL | 1589167 | | | Belize | Central America | 241000 | | | | Panama | Central America | 2856000 | | | | Costa Rica | Central America | 4023000 | | | | Nicaragua | Central America | 5074000 | | | | El Salvador | Central America | 6276000 | | | | Honduras | Central America | 6485000 | | | | Guatemala | Central America | 11385000 | | | | Mexico | Central America | 98881000 | | | | NULL | Central America | NULL | 16902625 | | | Saint Pierre and Miquelon | North America | 7000 | | | | Greenland | North America | 56000 | | | | Bermuda | North America | 65000 | | | | Canada | North America | 31147000 | | | | United States | North America | 278357000 | | | | NULL | North America | NULL | 61926400 | | | AVG Total | NULL | NULL | | 13053865 | +----------------------------------+-----------------+------------+----------+----------+ 41 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

With specifically placed NULL's and spaces in the SQL, the calculated chart can be created in such a way to bring out the data in a easy to read format all in one SQL statement. _________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-8

MySQL Developer Techniques

10.4.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

287

Thi

Chapter 10:Creating Reports

Quarterly Reports A common report concerning dates revolves around the idea of producing data for quarters. In many business environments, the quarters are connected to the quarters of the calendar year; January through March constitutes the first quarter, April through May constitutes the second quarter and so on. Typically, data is not stored on a quarterly or even monthly basis; rather, business events are recorded on a daily basis, storing the full date (sometimes including even the time of day) when the event occurred. For example, sales of items in a supermarket are typically recorded by storing the product code and time of day. SaleDate

SalePrice

SaleCost

2007-10-15 2007-11-07

$23,000.00 $17,450.00

$14,300.00 $14,465.00

2007-12-14 2008-01-06 2008-02-22

$21,456.00 $14,525.00 $16,395.00

$16,540.00 $8,780.00 $10,700.00

2008-03-17 2008-04-29

$27,560.00 $18,750.00

$22,420.00 $14,500.00

2008-05-12 2008-05-22

$21,500.00 $19,600.00

$17,300.00 $12,400.00

e

bl a r fe

s

an r t n

no a s a 2008-06-13 $14,100.00 $9,500.00 h n) ideฺ v ฺ The chart above (a used car sales report) represents a used car and illustrates what the underlying msales u odata, G c data for a quarterly report might look like. In the sample there are dates for only three quarters ฺ t n December, the first quarter of 21 October represented; the last quarter of 2007 which includes through e s d t 2008 which includes January through March and the second tu quarter of 2008 which includes April to stopgroup S June. The following SQL could be used the data into a quarterly report: @ s i n h a t r ฺt asu'Sales se Year', n mysql> SELECT YEAR(SaleDate) o o to SaleDate, NULL)) THEN SalePrice END) AS Q1, -> SUM(IF(QUARTER(SaleDate)=1, m ( e -> SUM(IF(QUARTER(SaleDate)=2, SaleDate, NULL)) THEN SalePrice END) AS Q2, an icens r -> SUM(IF(QUARTER(SaleDate)=3, SaleDate, NULL)) THEN SalePrice END) AS Q3, T l t -> SUM(IF(QUARTER(SaleDate)=4, SaleDate, NULL)) THEN SalePrice END) AS Q4 e-> FROM carsales y u g -> GROUP BY 'Sales Year';

N

This SQL would produce a chart similar to the one below that shows the total sales (sale price) for each quarter of the year the sale took place. NULL's are placed in those columns where there is no data for that quarter of the year. +------------+-------+-------+------+-------+ | Sales Year | Q1 | Q2 | Q3 | Q4 | +------------+-------+-------+------+-------+ | 2007 | NULL | NULL | NULL | 61906 | | 2008 | 58480 | 73950 | NULL | NULL | +------------+-------+-------+------+-------+ 2 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-9

MySQL Developer Techniques

Chapter 10:Creating Reports

10.4.1. Cross Tab Reports

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

288

Thi

Even though the output of the quarterly report presented earlier is acceptable, there may be times that the data itself may need to be presented to highlight attention to the quarters (versus the years). The following SQL statement provides quarterly data with the quarters highlighted: mysql> SELECT QUARTER(SaleDate) AS Quarter, -> SUM(IF(YEAR(SaleDate)=2007, SalePrice, NULL)) AS '2007', -> SUM(IF(YEAR(SaleDate)=2008, SalePrice, NULL)) AS '2008' -> FROM carsales GROUP BY Quarter; +---------+-------+-------+ | Quarter | 2007 | 2008 | +---------+-------+-------+ | 1 | NULL | 58480 | | 2 | NULL | 73950 | | 4 | 61906 | NULL | +---------+-------+-------+ 3 rows in set (#.## sec)

e

bl a r fe

s

an r t n

This method of creating a cross tab report of this nature is more hard coded (having to actually enter in the years) than the standard quarterly reports but can add value depending on the requirements of the end user. In addition, for those quarters where no data was found (such as Q3), the standard quarterly report still reflects that information versus this type of approach that did not.

no a s a h Column totals 289 n)yearlyidtotals eฺ can be calculated to v ฺ By providing the WITH ROLLUP clause to the end of the last query, om t Gu go along with the quarters displayed: c ฺ 21 den s t p Stu mysql> SELECT QUARTER(SaleDate) AS s Quarter, -> SUM(IF(YEAR(SaleDate)=2007, SalePrice, AS '2007', @ is NULL)) n h -> SUM(IF(YEAR(SaleDate)=2008, SalePrice, NULL)) AS '2008' a t r e WITH ROLLUP; ฺt BY Quarter -> FROM carsales GROUP s n u o +---------+-------+--------+ o to m | Quarter | 2007 | 2008 e | ( +---------+-------+--------+ ns | an ic| e58480 r | 1 | NULL T t 2 | NULLl | 73950 | | ye u | NULL | Ng|| NULL4 || 61906 61906 | 132430 | +---------+-------+--------+ 4 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-10

MySQL Developer Techniques

10.5.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

290

Thi

Chapter 10:Creating Reports

SQL Bar Chart Sometimes, information can be better displayed using a bar chart instead of a listing of the raw data. For example, the following displays the population values for the regions in the African continent::

mysql> SELECT Region, SUM(Population) -> FROM Country -> WHERE Continent='Africa' -> GROUP BY Region; +-----------------+-----------------+ | Region | SUM(Population) | +-----------------+-----------------+ | Central Africa | 95652000 | | Eastern Africa | 246999000 | | Northern Africa | 173266000 | | Southern Africa | 46886000 | | Western Africa | 221672000 | +-----------------+-----------------+ 5 rows in set (#.## sec)

291

e

bl a r fe

s

an r t n

The following SQL statement would convert these values into a bar chart for a visual representation of the data:

no a mysql> SELECT Region, REPEAT('#',(SUM(Population)/5000000)) ASs'Population Chart' a -> FROM Country h ) -> WHERE Continent='Africa' n ideฺ v ฺ -> GROUP BY Region; om t Gu +-----------------+---------------------------------------------------+ c ฺ | Region | Population Chart | 21 den s +-----------------+---------------------------------------------------+ t p Stu | Central Africa | ################### | s @ s | Eastern Africa | ################################################# | i n h a t | Northern Africa | ################################### | r t e ฺ s n | Southern Africa | ######### | o to u | Western Africa |o############################################ | (m nse +-----------------+---------------------------------------------------+ n(#.## 5 rows in rset sec) a e c T i l et y u g This of course does not replace the more detailed graphs that can be created in other tools; however,

N

for a quick representation of the data within MySQL itself it can be very useful.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-11

MySQL Developer Techniques

Chapter 10:Creating Reports

Lab 7-A

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this hands on lab, you will create a cross tab report for car sales that focuses on the actual days of the week that the cars were sold.

Thi

Lab Preparation To perform the following lab a CarSales table needs to be created in the test database. The following SQL statement builds the structure for the CarSales table: mysql> -> -> -> ->

CREATE TABLE CarSales ( SaleDate date NOT NULL DEFAULT '0000-00-00', SalePrice decimal(8,2) NOT NULL DEFAULT 0.0, SaleCost decimal(8,2) NOT NULL DEFAULT 0.0 );

e

bl a r fe

s

an r t n

no a s a h ฺ n) VALUES mysql> INSERT INTO CarSales (SaleDate, SalePrice, SaleCost) e v ฺ d i -> ('2007-10-15',23000.00,14300.00), om t Gu c -> ('2007-11-07',17450.00,14465.00), ฺ n -> ('2007-12-14',21456.00,16540.00), 21 e s d t -> ('2008-01-06',14525.00,8780.00), sp s Stu -> ('2008-02-22',16395.00,10700.00), @ i n -> ('2008-03-17',27560.00,22420.00), h a t r t -> ('2008-04-29',18750.00,14500.00), e nฺ us -> ('2008-05-12',21500.00,17300.00), o o o t -> ('2008-05-22',19600.00,12400.00), (m nse -> ('2008-06-13',14100.00,9500.00); n a r ce T i l t 1. eSales per week, per day guy1. It is fairly common that sales are not evenly distributed over all days of the week. Create a query to report The following SQL statement inserts the data into the CarSales table that will be used in the following lab:

N

the number of cars sold as a crosstab, having one row for each year and one column for each weekday. Hint: Examine the quarterly CarSales report shown previously. Use the WEEKDAY function to obtain the day for a particular date. 2. The resultant table should resemble the following MySQL output: +------+------+------+------+------+------+------+------+ | Year | Mon | Tue | Wed | Thu | Fri | Sat | Sun | +------+------+------+------+------+------+------+------+ | 2007 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | | 2008 | 2 | 1 | 0 | 1 | 2 | 0 | 1 | +------+------+------+------+------+------+------+------+

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-12

MySQL Developer Techniques

10.6.

Decision Tables Decision tables are used to lay out in tabular form all possible situations which a business decision may encounter and to specify which action to take in each of these situations. The following demonstrates a decision table for a world wide business focused on selling financial products.

292

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Chapter 10:Creating Reports

Rules

Conditions

Actions

2

3

4

5

6

Avg. Life Expectancy >= 65

Y

Y

Y

N

N

N

GNP per citizen >= 25M

Y

N

N

Y

N

N

GNP per citizen >= 250K

Y

Y

N

Y

Y

N

Offer Retirement Options

X

Offer Life Insurance

X

X

X

X

Offer Banking Services

X

X

X

e

bl a r fe

X

ns a r t - 65 years n • Average life expectancy - If the average life expectancy is equal to or greater than o n rules. of age for a given country, then the financial advisor will review the firstathree s a per citizen is equal to or • GNP per citizen (>= 25M) - If the gross national product (GNP) h ) ฺ the financial officer n idthen greater than 25,000,000 dollars (or an equivalent dollarvexchange), e ฺ will review the first and fourth rule to determinem products u should be offered to the o bywhich G c customer. The GNP per citizen can be determined dividing the country's GNP by the total ฺ t 21 den population of that country. s t p theSfacttuthat the majority of countries have a rather • GNP per citizen (>= 250K) -sDue to s @ i must review the first, second, fourth and fifth nthe financialthadvisor small GNP per citizen, a r t e ฺper citizenusis greater than (or equal to) 250,000 dollars (or an equivalent rules if the GNP onand dollar exchange) than 25,000,000 dollars. This time the financial advisor will only o ocondition tless e need ( tom review the first based on the fact that the second condition could not apply. n ens a r Based T on whichlicconditions apply to the country in question, the financial advisor has up to three t e products that should be recommended to the customer: uy

293

g

N Thi

1

There are three conditions that the financial advisor will use to determine which products they can offer to citizens of a country:

•

Retirement options - These would include packages that are most appropriate where countries with an average life span equal to or greater than 65 years of age. For countries with an earlier average life span, the potential for customers living to retirement is not conducive for retirement products.

•

Life insurance - These would include packages that are most appropriate in countries where each individual has a higher possibility (based on the GNP per citizen) of having the resources to purchase (or the need to purchase) life insurance.

•

Banking services - These would include packages where the citizens may have some additional funds that can be used for saving and investing (based on the GNP per citizen). In countries where the GNP per citizen is less than 250,000 dollars (or an equivalent dollar exchange), the likelihood of the citizens having funds beyond everyday living for savings or investment purposes would be improbable.

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-13

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

294

Thi

Implementing with SQL Now that the decision table has been made, it is time to implement the conditions and actions into SQL statements. The first step is to work through the conditions: mysql> -> -> ->

295

Chapter 10:Creating Reports

SELECT Name, IF(LifeExpectancy >= 65, 'Rules 1 thru 3', 'Rules 4 thru 6' ) AS Action FROM Country;

This SELECT statement tests the first condition determining if the average life expectancy is greater then or equal to 65. If the condition is met, the text will display that 'Rules 1 thru 3' have been met, and if not, the text will display 'Rules 4 thru 6'. mysql> SELECT Name, IF(LifeExpectancy >= 65, -> IF((GNP*10000000000/Population) >= 25000000, -> 'Offer Retirement, Insurance and Banking', -> 'Rules 2 thru 3' -> ), -> IF((GNP*10000000000/Population) >= 25000000, -> 'Offer Insurance and Banking', -> 'Rules 5 thru 6' -> ) ->) AS Action FROM Country;

e

bl a r fe

s

an r t n

no a s a h ) statement ฺ which handled nSELECT This SELECT statement adds the second condition to the original e v ฺ d i the first condition. These nested IF functions, if met will m displayu'Offer Retirement, Insurance oeither G is greater then or equal to c ฺ and Banking' (which is rule 1) if the average life expectancy for the country t 296 1 n 2 e life expectancy for the country is 65 or 'Offer Insurance and Banking' (which is 4) if thedaverage ptsrule tu are not met for the first nested IF or less then 65. 'Rule 2 thru 3' will be displayed if theS conditions s @met forththeissecond IF function. 'Rule 5 thru 6' if the conditions are not n a r ฺt use >= 65, n mysql> SELECT Name, IF(LifeExpectancy o o to -> IF((GNP*10000000000/Population) >= 25000000, m ( e s -> 'Offer Retirement, Insurance and Banking', n a icen -> TrIF((GNP*10000000000/Population) l Insurance and Banking',>= 250000, ->t 'Offer e 'Offer Insurance' uy -> g -> ) N -> ), -> IF((GNP*10000000000/Population) >= 25000000, -> 'Offer Insurance and Banking', -> IF((GNP*10000000000/Population) >= 250000, -> 'Offer Banking', -> 'No offer suggested' -> ) -> ) -> ) AS Action FROM Country;

This SELECT is the final statement that takes into account all possible conditions for the decision table and produces an output that would provide the financial advisor with the options they should suggest to any potential customers based on the country they are citzens of. _________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-14

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

297

Thi

Chapter 10:Creating Reports

Here is an example of the script executed with two different countries: Monaco and Haiti. mysql> SELECT Name, IF(LifeExpectancy >= 65, -> IF((GNP*10000000000/Population) >= 25000000, -> 'Offer Retirement, Insurance and Banking', -> IF((GNP*10000000000/Population) >= 250000, -> 'Offer Insurance and Banking', -> 'Offer Insurance' -> ) -> ), -> IF((GNP*10000000000/Population) >= 25000000, -> 'Offer Insurance and Banking', -> IF((GNP*10000000000/Population) >= 250000, -> 'Offer Banking', -> 'No offer suggested' -> ) -> ) -> ) AS Action FROM Country WHERE Name = 'Monaco'; +--------+-----------------------------------------+ | Name | Action | +--------+-----------------------------------------+ | Monaco | Offer Retirement, Insurance and Banking | +--------+-----------------------------------------+ 1 row in set (#.## sec)

e

s

an r t n

o

an s a h ) n ideฺ mysql> SELECT Name, IF(LifeExpectancy >= 65, v ฺ -> IF((GNP*10000000000/Population) >= 25000000, om t Gu c -> 'Offer Retirement, Insurance and Banking', ฺ 1>= 250000, n -> IF((GNP*10000000000/Population) 2 e s d t -> 'Offer Insurance and Banking', sp s Stu -> 'Offer Insurance' @ i n -> ) h a t r t -> ), e nฺ us -> IF((GNP*10000000000/Population) >= 25000000, o o o t -> 'Offer Insurance and Banking', m -> IF((GNP*10000000000/Population) >= 250000, se n ( eBanking', n -> Tra 'Offer c i ->t 'Nol offer suggested' e y -> ) gu -> )

N

bl a r fe

-> ) AS Action FROM Country WHERE Name = 'Haiti'; +-------+---------------+ | Name | Action | +-------+---------------+ | Haiti | Offer Banking | +-------+---------------+ 1 row in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-15

MySQL Developer Techniques

10.7.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

298

Chapter 10:Creating Reports

Materialized Views MySQL does not have materialized views; however, many of the features of materialized views can be completed with summary tables. The term "summary tables" can be misleading. Summary tables, in relation to the discussion here, includes those tables that summarize data from other sources (most likely other tables). The following is an example of query that could be considered a summary table:

mysql> -> -> -> ->

CREATE TABLE MV_Country AS SELECT Country.Region, MAX(Country.LifeExpectancy) AS MaxLifeExpectancy, COUNT(DISTINCT CountryLanguage.Language) AS LanguagesSpoken FROM Country,CountryLanguage WHERE Country.Code=CountryLanguage.CountryCode GROUP BY Region;

mysql> SELECT Region, MaxLifeExpectancy, LanguagesSpoken FROM MV_Country;

299 +---------------------------+-------------------+-----------------+ | Region | MaxLifeExpectancy | LanguagesSpoken | +---------------------------+-------------------+-----------------+ | Australia and New Zealand | 79.8 | 11 | | Baltic Countries | 69.5 | 8 | | British Islands | 77.7 | 4 | ... | Southern and Central Asia | 71.8 | 54 | | Southern Europe | 83.5 | 22 | | Western Africa | 76.8 | 65 | | Western Europe | 79.6 | 21 | +---------------------------+-------------------+-----------------+ 24 rows in set (#.## sec)

e

bl a r fe

s

an r t n

T

no a s a h n) ideฺ v ฺ om t Gu c ฺ 1 for theenaverage life expectancy by region, while In this example, the world.Country table is 2 squeried dfor the number of languages spoken in each t the joined world.CountryLanguage p table is queried tuquery s S region. The data has been summarized and from this more specific data can be retrieved. By @ s i n h having a summary table of thea data, accessing this summary data can cost less performance wise when tr se t ฺ user require access to thisndata. o to u Different viewsm of o the same data 300 seusing a join query, end users may need to use the data in different ways. One n (summarized With theadata n r e c Tuser may beliconcerned end with the average life span and will wish to sort the data showing the shortest t e life span countries first. of course, the query can be changed to sort on the y u AvgLifeExpectancy column,Now g but that could cause problems for another user that is more concerned with the number of languages spoken. The answer to this dilemma is to give each user a table of the hi N summarized data that they can perform more specific queries on. Updating tables A dilemma associated with "summary tables" is how to keep them accurate and up to date; if that is necessary. An answer to this dilemma is the use of triggers. When data that makes up the summary table is updated or changed, triggers can be set up to catch these events and create a new table for the users to access. Triggers can have an effect on overall performance of the queries that update or alter the tables which must be taken into account when setting up such processes. Other less performance heavy options may include nightly batch processes that include deleting and recreating the tables based on the most current data at that time (if up to date data is not absolutely necessary).

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-16

MySQL Developer Techniques

10.8.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

301

Thi

Chapter 10:Creating Reports

Producing Sequential or Missing Data One of the ways to produce sequential or missing data is by utilizing an integer table. An integer table is a single column table that contains a sequential list of numbers. The syntax could simply be as follows for creating the table:

mysql> CREATE TABLE int_table (int_col INT PRIMARY KEY NOT NULL AUTO_INCREMENT);

Filling the table with data Once the table is created, it is necessary to fill the table with data. The number of rows that should be inserted, should be much larger than the number of rows in the largest table that is located in the database that this table will support. For example, in the world database, the table with the greatest number of rows is the city table with approximately 4,000 records. So in this case, the integer table should contain somewhere around 8,000 records to ensure that there is room for the city table to grow. Now of course, it is possible to create the max number of integer rows into the table (4,294,967,295 unsigned); however, if none of the tables in the database will ever reach that number there is no reason to waste even this small amount of space with unneeded rows. It would be quite easy to add additional rows at a later date. 302

•

e

bl a r fe

s

an r t n

Exponential growth - One of the ways to fill the integer table with values is to use the table itself in an exponential fashion to grow itself. The following SQL shows how this would work:

no a s a h mysql> INSERT INTO int_table VALUES (NULL); n) ideฺ v ฺ om t Gu mysql> SELECT int_col FROM int_table; c ฺ +---------+ 21 den s t | int_col | sp s Stu +---------+ @ i | 1 | n h a t r +---------+ t e 1 row in set (#.## sec) onฺ to us o (m nse n Adding a NULL value into the int_col column will result in the next sequential number being a r e c T i l added to the table. In this case, it was the first record created and thus the integer 1 was t e y added. gu

N

mysql> INSERT INTO int_table SELECT NULL FROM int_table; mysql> INSERT INTO int_table SELECT NULL FROM int_table; mysql> INSERT INTO int_table SELECT NULL FROM int_table; mysql> SELECT * FROM int_table; +---------+ | int_col | +---------+ | 1 | ... | 8 | +---------+ 8 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-17

MySQL Developer Techniques

Chapter 10:Creating Reports

By executing the INSERT statement above that inserts a NULL value a number of times equivalent to the rows that are located in the int_table, the number of records in the int_table grows exponentially.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

303

Thi

•

Stored procedure - Another way to fill the integer table is to create a stored procedure. In the following stored procedure, the user enters the number of records they wish to have added to the int_table:

mysql> DELIMITER // mysql> CREATE PROCEDURE fill_int (int_max INT) -> BEGIN -> DECLARE x INT DEFAULT 0; -> WHILE x < int_max DO -> INSERT INTO int_table VALUE (NULL); -> SET x = x + 1; -> END WHILE; -> SELECT COUNT(*) FROM int_table; -> END // Query OK, 0 rows affected (#.## sec)

e

bl a r fe

s

an r t n

no a s mysql> CALL fill_int(8000); a h +----------+ n) ideฺ | COUNT(*) | v ฺ +----------+ om t Gu c | 8000 | ฺ 21 den +----------+ s t 1 row in set (#.# sec) sp s Stu Query OK, 0 rows affected (#.# sec) @ i n h a t r t e s input provided (in this case 8,000) and executes an INSERT The stored procedure, uthe onฺ ttakes o o statement against e the int_table that number of times. After the WHILE loop is completed, the (m s n stored procedure performs a SELECT query against the table to count how many rows the n a r table contains. ce This process allows for more control over how many rows are added, but the T i l t e cost in time (for the procedure to run) is greatly increased. guy mysql> DELIMITER ;

N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-18

MySQL Developer Techniques

Chapter 10:Creating Reports

Lab 7-B

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this lab you create an integer table and then use that table to produce non-integer sequential outputs.

Thi

1.

1. Create and load the integer table Using the world database, create the following table:

mysql> CREATE TABLE int_table (int_col INT PRIMARY KEY NOT NULL);

2.

This will create the integer table that will be used throughout the remainder of this lab. Insert the values 0 through 9 into the int_table using the following SQL:

INSERT INTO int_table VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);

This will populate the integer table with the numbers 0 through 9 for use in the views to be created. 1.

2. Create the English alphabet view Create the following view that will list the 26 letters of the English alphabet:

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e ฺ us onthat This will create o a view will display the 26 letters of the English alphabet. o t (m senames by first letter 3. Count the country n n a r e T the following 1. Perform lic SQL query to count the names of the countries (using the first letter of the country t e name): guy CREATE VIEW abc(int_col) AS SELECT SUBSTRING('ABCDEFGHIJKLMNOPQRSTUVWXYZ' FROM 10*tbl1.int_col+tbl2.int_col FOR 1) AS Alphabet FROM int_table AS tbl1 CROSS JOIN int_table AS tbl2 WHERE 10*tbl1.int_col+tbl2.int_col BETWEEN 1 AND 26 ORDER BY Alphabet;

N

mysql> SELECT LEFT(Name,1) AS Alpha, -> COUNT(LEFT(Name,1)) AS Total FROM -> Country GROUP BY LEFT(Name,1);

Each letter of the English alphabet is represented by the country names with the exception of the letter X: +-------+-------+ | Alpha | Total | +-------+-------+ | A | 15 | ... | W | 2 | | Y | 2 | | Z | 2 | +-------+-------+ 25 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-19

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

2.

Thi

Chapter 10:Creating Reports

Perform the following SQL query to have every letter of the English alphabet displayed (including X) with the total number of countries that start with that letter:

SELECT abc.int_col AS Alpha, COUNT(LEFT(Name,1)) AS Total FROM abc LEFT OUTER JOIN Country ON abc.int_col = LEFT(Name,1) GROUP BY abc.int_col;

Each letter of the English alphabet is represented by the country names including the letter X: +-------+-------+ | Alpha | Total | +-------+-------+ | A | 15 | | B | 20 | | C | 22 | ... | V | 5 | | W | 2 | | X | 0 | | Y | 2 | | Z | 2 | +-------+-------+ 26 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-20

MySQL Developer Techniques

Chapter 10:Creating Reports

10.8.1. Using the integers table to fill in missing data

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

304

Thi

Now that the integers table has been created, it is time to demonstrate how this can be used to fill in missing data for another table. In the following example, a copy of the world.city table has been created (city2) with records 6 through 9 deleted: mysql> SELECT * FROM city2 LIMIT 10; +----+----------------+-------------+---------------+------------+ | ID | Name | CountryCode | District | Population | +----+----------------+-------------+---------------+------------+ | 1 | Kabul | AFG | Kabol | 1780000 | | 2 | Qandahar | AFG | Qandahar | 237500 | | 3 | Herat | AFG | Herat | 186800 | | 4 | Mazar-e-Sharif | AFG | Balkh | 127800 | | 5 | Amsterdam | NLD | Noord-Holland | 731200 | | 10 | Tilburg | NLD | Noord-Brabant | 193238 | | 11 | Groningen | NLD | Groningen | 172701 | | 12 | Breda | NLD | Noord-Brabant | 160398 | | 13 | Apeldoorn | NLD | Gelderland | 153491 | | 14 | Nijmegen | NLD | Gelderland | 152463 | +----+----------------+-------------+---------------+------------+ 10 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a Using the integer table (in this case, the 100k view created earlier will be used), s the following OUTER a 305 h JOIN query will fill in the missing integers: n) ideฺ v ฺ mysql> SELECT 100k.int_col AS ID, city2.Name, city2.CountryCode om t Gu FROM 100k c ฺ -> LEFT OUTER JOIN 21 den -> city2 ON 100k.int_col = city2.ID s t -> LIMIT 1,10 sp s Stu +----+----------------+-------------+---------------+------------+ @ i n | tDistrict h | ID | Name | CountryCode | Population | a r t e ฺ +----+----------------+-------------+---------------+------------+ o|nAFG to us | Kabol | 1 | Kabul | 1780000 | o | 2 | Qandahar(m | AFG | Qandahar | 237500 | e n | 3 | Herat |sAFG | Herat | 186800 | n a r e | 4 | Mazar-e-Sharif | AFG | Balkh | 127800 | c T i l | 5e|t Amsterdam | NLD | Noord-Holland | 731200 | u| y6 | NULL | NULL | NULL | NULL | g | NULL | NULL | NULL | N | 7 | NULL | 8 | NULL | NULL | NULL | NULL | | 9 | NULL | NULL | NULL | NULL | | 10 | Tilburg | NLD | Noord-Brabant | 193238 | +----+----------------+-------------+---------------+------------+ 10 rows in set (#.## sec)

In this OUTER JOIN query, the int_col column from the 100k view takes the place of the ID column of the city2 table along with displaying the remaining columns of the city2 table (Name, CountryCode).

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-21

MySQL Developer Techniques

Chapter 10:Creating Reports

Lab 7-C

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

In this lab you will use the knowledge learned up to this point to produce cross tab reports that take advantage of the integer table idea presented earlier along with creating a SQL bar chart to represent the data queried.

Thi

1.

2.

1. Independent Governments of the second half of the 20th century Make a report that produces one row for each year of the last 50 years of the previous century. Make a comma separated list of all countries that became independent in that particular year, and also list the total number of countries. Hint: Use an integer table to deal with missing years. The resultant table should resemble the following MySQL output:

+------+---------------+-----------------------------+ | year | num_countries | GROUP_CONCAT(country.name) | +------+---------------+-----------------------------+ | 1951 | 2 | Libyan Arab Jamahiriya,Oman | | 1952 | 0 | NULL | | 1953 | 2 | Laos,Cambodia | | 1954 | 0 | NULL | | 1955 | 1 | Germany | | 1956 | 3 | Sudan,Tunisia,Morocco | | 1957 | 2 | Ghana,Malaysia | ... | 1998 | 0 | NULL | | 1999 | 0 | NULL | | 2000 | 0 | NULL | +------+---------------+-----------------------------+ 50 rows in set (#.## sec)

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e 3. Create an SQL bar chart s of countries that became independent in the last 50 years of the onฺto tlistotheunumber previous century. o (mtable nshould 4. The resultant se resemble the following MySQL output: n a r e T lic t +------+--------------------+ e yyear | num_countries | gu|+------+--------------------+

N

| 1951 | ** | | 1952 | | | 1953 | ** | | 1954 | | | 1955 | * | | 1956 | *** | ... | 1990 | *** | | 1991 | ****************** | | 1992 | * | | 1993 | *** | | 1994 | * | ... | 2000 | | +------+--------------------+ 50 rows in set (#.## sec)

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-22

MySQL Developer Techniques

10.9.

Chapter 10:Creating Reports

Summary In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

306

Thi

•

Use functions to calculate multiple conditions

•

Create a report with values and subtotals for each group along with final rollup

•

Create aggregate reports against dates in the data

•

Create cross tab reports

•

Create a bar chart with SQL

•

Create a decision table to avoid hard wiring of logic in the application code

•

Create summary tables with temporary tables

•

Produce sequential or missing data

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

_________________________________________________________________________________________________ _________________________________________________________________________________________________ _________________________________________________________________________________________________

10-23

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

CHAPTER 11

e

s

an r t n

o

CONCLUSIONhas a n n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

Chapter 11: Conclusion

11 CONCLUSION

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

11.1 Course Objectives

Thi

308

This course was designed for experienced database developers who wished to learn additional skills required to create complex queries and efficient structures while improving the performance of the database by learning to create and write queries that are optimized based on the data in the database. Now that you have completed this course, you should now be able to: •

Utilize the multiple indexing options available to create better response times of query executions

•

Optimize queries that are responsible for searching text fields

•

Optimize queries that search date fields

•

Optimize queries that perform calculations and aggregations

•

Create summary tables that utilize triggers, event scheduler and table merges

•

Optimize queries that retrieves data from more than one table

e

bl a r fe

s

an r t n

309

o n a • Create tables that utilize tree and hierarchical structures s a h • Utilize temporary tables to improve query performance n) eฺ v ฺ d i om t Gu • Explain the affect locking levels have on querycperformance ฺ n 21 dprecise e • Create report queries that provide p thets end userstwith and organized data u s S @ this n a r ฺt use n o o to m ( e an icens r T l et

uy g N

11-1

MySQL Developer Techniques

Chapter 11: Conclusion

11.2 Training and Certification Website View the course catalog, schedule, certification program information, venue information (and more) on the Training and Certification website: http://www.mysql.com/training/ .

310

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

24H

e

bl a r fe

s

Thi

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Topic Menu

an r t n

uy g N

11-2

MySQL Developer Techniques

Chapter 11: Conclusion

ƒ Overview – The “home page” for Training information. Links to all other training information. ƒ Certification – Description of current certification program. Links to exam registration and more information.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

ƒ Curriculum Paths – The categories and order in which training courses should be taken.

Thi

ƒ Catalog – Comprehensive course catalog. Links to more information on courses. ƒ Schedule – Schedule of courses, listed by title. Links to course registration and more information. ƒ Delivery Options – The available options for training delivery. ƒ Savings and Promotions – Information about training bundles and credits. ƒ Training Bundles – Available bundles and pricing. ƒ Training Partners – List and links of MySQL training partners. ƒ Testimonials – Training endorsements.

e

ƒ FAQ – Frequently asked questions and links to more other training information.

bl a r fe

s

an r t n

Additional training information, including Articles, White Papers and Web Seminars, can be found on the Community web page: http://dev.mysql.com/ .

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

11-3

MySQL Developer Techniques

Chapter 11: Conclusion

11.3 Course Evaluation We Need Your Evaluation!

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

311

Thi

Please take the time to complete a course evaluation to let us know about your experience. We are continuously making improvements to our courses and our overall training program. And we count on the feedback of our students to assist in making the right choices. Please follow the instructions given by your instructor regarding completion of the evaluation form.

Thank you in advance for taking the time to give us your opinions!

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

11-4

MySQL Developer Techniques

Chapter 11: Conclusion

11.4 THANK YOU! We appreciate your attendance at this course. Congratulations on your completion. We hope that your training needs regarding this course topic have been fully met. Don’t hesitate to contact us, if you have any training questions:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

312

Thi

Website: http://www.mysql.com/training/ Email: [email protected] Phone: USA Toll Free: 1-866-697-7522 Worldwide: 1-208-514-4780

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

11-5

MySQL Developer Techniques

Chapter 11: Conclusion

11.5 Q&A Session 313

You may record any questions and answers that are shared at this point in the class, in the space provided below.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

___________________________________________________________________________________________

Thi

___________________________________________________________________________________________ ___________________________________________________________________________________________ ___________________________________________________________________________________________ ___________________________________________________________________________________________ ___________________________________________________________________________________________

e

N

bl a r ___________________________________________________________________________________________ fe s n ___________________________________________________________________________________________ tra n no ___________________________________________________________________________________________ a s a h ___________________________________________________________________________________________ n) ideฺ v ฺ om t Gu ___________________________________________________________________________________________ c ฺ 1 n 2 e s d t ___________________________________________________________________________________________ sp s Stu @ i n h ___________________________________________________________________________________________ a t r t e nฺ us o o o ___________________________________________________________________________________________ t (m nse n a r ___________________________________________________________________________________________ ce T i l t ye gu___________________________________________________________________________________________ ___________________________________________________________________________________________ ___________________________________________________________________________________________ __________________________________________________________________________________________ ___________________________________________________________________________________________ ___________________________________________________________________________________________

11-6

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

APPENDIX A

e

s

bl a r fe

o

an r t n

uy g N

an INSTALLATION s a h ) n ideฺ v ฺ PROCEDURES om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

Appendix A: Installation Procedures

Appendix A - Installation Procedures A.1

MySQL Server installation on Microsoft Windows

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

For the purposes of this course, you will download the 5.1 Generally Available release, Essentials Version:

Thi

1.

Go to the MySQL web site downloads page: http://dev.mysql.com/downloads/

2.

Find section entitled: “MySQL Community Server”

3.

Find the section called “Upcoming Releases” and click the link for: ”MySQL 5.1”

4.

Find the “Essentials” version of the appropriate platform and click on the link called “Pick a mirror”

5.

The registration at the top of the page is optional, so to skip it select the link at the bottom of the page that reads “No thanks, just take me to the downloads!”. You will not perform registration during the course.

6.

Scroll down the page to the section showing the “Mirrors in: ”. Find the location closest to you and click on the link next to it called “HTTP”:

7.

A window will pop up with the option to Run or Save.

8.

Select Run to download the software for installation. (If a window regarding security appears, click on Run to allow the download.)

s

an r t n

no a s 10. Select a Typical installation. a h ) be as follows: The wizard is ready to install and indicates that the install directoryn will eฺ v ฺ d i “C:\Program Files\MySQL\MySQL Server 5.1” om t Gu c ฺ n 11. Start the install by clicking on the Install button.21 e s d t u will pop up. After the files are installed, the “MySQL.com twindow sp Sign-Up” S @ s i 12. Select Skip Sign-Up, and click onn the Next> h button. a t r t e 13. In the “Wizard Completed” make s sure that check box labeled “Configure the MySQL Server now” is u onฺ window, selected. Then, clickoon the Finish button. o t m Instance (Server e s The “MySQL Configuration wizard” will appear. n a icen r T l button to perform the configuration. 14. Click t on the Next> e y gu15. Select “Detailed Configuration”, then click the Next> button. 9.

N

e

bl a r fe

The Windows Setup wizard will appear. Click on the Next> button to perform the set-up.

16. Select “Developer Machine”, then click the Next> button.

17. Select “Multifunctional Database”, then click the Next> button. 18. Do not change any “InnoDB Tablespace Settings” at this time, and click the Next> button. 19. Select “Decision Support”, then click the Next> button. 20. Select “Enable TCP/IP Networking” and “Enable Strict Mode”, then click the Next> button. 21. Select “Standard Character Set”, then click the Next> button. 22. Select “Install As Windows Service” and “Include Bin Directory in Windows PATH”, then click the Next> button. 23. Select “Modify Security Settings”, then enter “root” as the new password. Then click the Next> button. Use of an “anonymous” account is not recommended, due to the lack of security. 24. Click on the Execute button. The wizard now shows that the configuration file was created successfully. 25. Click on the Finish button to complete.

A-1

MySQL Developer Techniques

A.2

Appendix A: Installation Procedures

MySQL Server installation on Linux

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

For the purposes of this course, you will download the 5.1 Release Candidate release, Essentials Version:

Thi

1.

Go to the MySQL web site downloads page: http://dev.mysql.com/downloads/

2.

Find section entitled: “MySQL Community Server”

3.

Find the section called “Upcoming Releases” and click the link for: ”MySQL 5.1”

4.

Download the latest binary tarball: (eg. mysql-5.1.26-rc-linux-i686-icc-glibc23.tar.gz)

5.

Unpack the tarball to /usr/local with (change /tmp/ to the proper path):

cd /usr/local

e

tar –zxvf /tmp/mysql-5.1.26-rc-linux-i686-icc-glibc23.tar.gz

6.

s

Add a symbolic link to the directory:

N

an r t n

o

an s a h 7. Install the system tables: ) n ideฺ v ฺ cd mysql om t Gu c ฺ 21 den s scripts/mysql_install_db t sp s Stu @ i n 8. Add a mysql system user and a group: h t r t e onฺ to us groupadd mysql o se n (m n useradd –gra mysql mysql e T lic t e gu9.y Change the owner of the data directory: ln –s mysql-5.1.26-rc-linux-i686-icc-glibc23.tar.gz mysql

bl a r fe

chown –R mysql:mysql data

10. Copy the startup script to /etc/init.d: cp support-files/mysql.server /etc/init.d/mysql

11. Start the script: /etc/init.d/mysql start

12. Edit the bashrc file and add this path: PATH=${PATH}:/usr/local/mysql/bin

A-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

A.3

Thi

Appendix A: Installation Procedures

MySQL 'world' database file installation 1.

Go to the MySQL web site “Documentation” page: http://dev.mysql.com/doc/

2.

Find section entitled: “Example Databases”

3.

Select the most appropriate method for download of the world database: Note: For the purposes of this course, you will not need to download the setup guide.

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

A-3

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

APPENDIX B

e

s

bl a r fe

o

an r t n

uy g N

MYISAM has a n n) ideฺ v ฺ LOCKING om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

Appendix B: MyISAM Locking

Appendix B - MyISAM Locking

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Table lock types The MyISAM storage engine protects against corruption of data by different threads updating in conflicting ways by using table-level locking. The MyISAM storage engine utilizes one of three types of locks depending on the request issued by the connecting thread: READ LOCAL SHARED

READ SHARED

WRITE EXCLUSIVE

Allows an INSERT to append only to the end of the data file. If the INSERT requires a push of data anywhere else but to the end of the data file (beginning or middle of the data file), the INSERT statement is delayed until the lock is released.

Activated when the actual data file (*.MYD), rather than the in-memory cache, is used to get information. When this lock is activated, all UPDATE, INSERT and DELETE statements are delayed until the lock is released.

This type of lock is activated whenever an UPDATE or DELETE request is submitted, or an INSERT is received that would require adding the data anywhere but to the end of the data file. This lock prevents any activity on the table until the lock is released.

T

ns a r t -are running against With the MyISAM storage engine's capability to perform INSERT operations while SELECT queries n o n that are responsible for the data (through the READ LOCAL lock type), it makes an excellent choice for applications aquery s logging continual activities while also giving end users the ability to run reports or other on the data a will always operations h contained. This is made possible by the fact that MyISAM knows that all new )records occur at the end of the ฺ else in the table. ndata from eanywhere v data file and thus there is no need to hold up SELECT statements that request ฺ d i om t Gu c ฺ Concurrent n 21 Inserts e s d t p SELECT tustatements to use the READ LOCAL lock regardless if With the MyISAM storage engine, it is possible tosforce S @ s there is room in the middle of the data file for the INSERT to fill. Whether an INSERT will be a concurrent insert is anstatus thi This r determined by the concurrent-insertฺtglobal variable. variable can be assigned by setting the mysqld -e s u onGLOBAL concurrent_insert or o SET concurrent_insert parameter. to m ( e s take any one of the following values: The concurrent_insert ncan an parameter r e • 0 - (Off) This tells the server to not allow any concurrent inserts. c T li t e • 1 - (Default) Enables concurrent insert for MyISAM tables that don't have gaps. y •g 2u - Enables concurrent inserts for all MyISAM tables. If the table has a hole and is in use by another thread the new hi Nrow will be inserted at the end of the table. If the table is not in use, MySQL obtains a normal WRITE lock and inserts Tables designed to Log Activity

e

bl a r fe

the new row into the hole.

A-1

MySQL Developer Techniques

Appendix B: MyISAM Locking

Priorities

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

When running a SQL statement that interacts with a MyISAM table, an individual user can determine the importance level that their individual statements have in regards to priority execution on the MySQL server. There are four levels of importance in relation to database actions:

Thi

Highest priority

Normal writes

Normal reads

Lowest priority

The database actions that this level of priority (or importance) include SELECT or INSERT commands that the user or application interfacing with the server has identified as needing to run before any other SQL statements waiting to be executed by the server.

If there are no high priority SQL statements waiting to be executed, the MySQL server will allow INSERT, DELETE and UPDATE commands to run first prior to any SELECT statements that are waiting to be executed by the server.

As stated, SELECT statements are at the bottom of the execution list (unless identified as high priority) when waiting for execution by the server.

The database actions at this level of priority will wait for all other SQL statements to execute prior to actually executing themselves. The end user of the application interfacing with the server must specifically tell the server that these SQL statements are considered low priority otherwise the normal writes and reads order will be implemented.

s

an r t n

no a s to move to the top of the list of HIGH_PRIORITY – This command gives the SQL statement being executed the ability a h any waiting SQL statements (from all threads) and can be used with the following SQL n) idstatements: eฺ v ฺ u HIGH_PRIORITY INSERT HIGH_PRIORITY om t GSELECT c ฺ 1 the INSERT n MySQL gives the SELECT a higher 2gives e MySQL stops any concurrent insert capabilities and s d t thasu executed priority than a statement that updates a statement complete control of the table until s thepstatement S @ s completely. table. This should only be used for i n h a t r queries that are very fast and must be t e ฺ > Priority levels s n done at once. A SELECT u o o o > When using the INSERT HIGH_PRIORITY statement, the server HIGH_PRIORITY query that is issued t m automatically overrides se low_priority_updates setting. while the table is locked for reading n ( anyeexisting n a r The low_priority_updates setting, when set to 1, forces all INSERT, runs even if there is an update statement T ic TABLE lLOCK t e UPDATE, DELETE, and statements to wait until there waiting for the table to be free. uypending SELECT or LOCK TABLEWRITE is no READ on the affected table. This HIGH_PRIORITY cannot be used with g N The following is a list of the MySQL priority commands that can be used:

variable previously was named sql_low_priority_updates. In addition, by default writing queries have a higher priority than reading queries, so by default there are two priority queues.

e

bl a r fe

SELECT statements that are part of a UNION.

A-2

MySQL Developer Techniques

Appendix B: MyISAM Locking

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

LOW_PRIORITY – This command places the SQL statement being executed at the bottom of the list of any waiting and new SQL statements (from all threads) and will only run when all waiting and new SQL statements have completed. This command can be used with the following SQL statements:

Thi

INSERT LOW_PRIORITY

UPDATE LOW_PRIORITY

DELETE LOW_PRIORITY

MySQL delays the execution of the statement (and stops any other work from the particular client) until no other clients are reading from the table. This includes any other clients that may access the table while the INSERT LOW_PRIORITY statement is waiting to execute. In an environment where there is a large number of clients that can be reading from the server at any one time this could cause the client to wait a very long time. This should normally not be used with MyISAM tables because doing so disables concurrent inserts.

Works identical to the INSERT LOW_PRIORITY statement and should also be avoided unless absolutely necessary.

Also works the same as the INSERT LOW_PRIORITY by not executing until all clients are done reading from the table. This can cause integrity issues in that another client can be viewing data that should be deleted immediately.

e

bl a r fe

ns a r t - queue forever, if Using LOW_PRIORITY with UPDATEs, INSERTs and DELETEs can cause the queries to wait innthe o there is no time of day when clients aren't reading from the table. an s a Miscellaneous h ) n ideฺ is not directly related to v ฺ INSERT DELAYED - The use of the INSERT DELAYED statement in SQL statements om tinGtheulocking section. When the INSERT locking issues in MyISAM, but has a bearing on many of theฺc discussions n into a buffer, and the client issuing the DELAYED statement is executed, the server puts the row2 or1 rows to be inserted e s d t p SIftuthe table is in use, the server holds the rows. When the INSERT DELAYED statement can then continues immediately. table is free, the server begins inserting rows, checking iperiodically to see whether there are any new read requests for @ s n h a t the table. If there are, the delayed row queue is suspended until the table becomes free again. Another major benefit of r e clients are bundled together ฺt usmany n using INSERT DELAYED is o that inserts from and written in one block. This is much o o t faster than performing many separate inserts. INSERT DELAYED is slower than a normal INSERT if the table is not (m e otherwise in use. There is also n thesadditional overhead for the server to handle a separate thread for each table for which n arows. Inicaddition, r e if the server crashes before the INSERT's have been actually inserted, they will be there are delayed T l t lost. y u Thise means that INSERT DELAYED should be used only when it is absolutely necessary. g N

A-3

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

APPENDIX C

e

s

bl a r fe

o

an r t n

uy g N

an QUIZ SOLUTIONS s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

Appendix C: Quiz Solutions

Appendix C - Quiz Solutions Quiz 2-A Solutions

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

1. SELECT d FROM t1 WHERE d = 10 mysql> EXPLAIN SELECT d FROM t1 WHERE d = 10\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: ref possible_keys: d key: d key_len: 5 ref: const rows: 1 Extra: Using where; Using index 1 row in set (#.## sec)

e

s

2. SELECT a FROM t1 WHERE a = 10

T

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a 3. SELECT r c FROM t1eWHERE c = 10 T lic t e y EXPLAIN SELECT c FROM t1 WHERE c = 10\G umysql> g *************************** 1. row *************************** id: 1 hi N mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: ref possible_keys: a key: a key_len: 5 ref: const rows: 1 Extra: Using where; Using index 1 row in set (#.## sec)

bl a r fe

select_type: SIMPLE table: t1 type: index possible_keys: NULL key: a key_len: 15 ref: NULL rows: 1 Extra: Using where; Using index 1 row in set (#.## sec)

C-1

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

4.

Appendix C: Quiz Solutions

SELECT d FROM t1 WHERE d = 10 ORDER BY a mysql> EXPLAIN SELECT d FROM t1 WHERE d = 10 ORDER BY a\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: ref possible_keys: d key: d key_len: 5 ref: const rows: 1 Extra: Using where; Using filesort 1 row in set (#.## sec)

5.

SELECT a FROM t1 WHERE a = 10 ORDER BY b mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 ORDER BY b\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: ref possible_keys: a key: a key_len: 5 ref: const rows: 1 Extra: Using where; Using index 1 row in set (#.## sec)

e

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n 6. SELECT a FROM t1 WHEREtraa= 10 ANDth b = 5 e ฺ us oanFROM o mysql> EXPLAIN SELECT t1 WHERE a = 10 AND b = 5\G o t (m nse 1. row *************************** *************************** n id: 1 a r SIMPLE ce T i select_type: l t t1 uye table: type: ref g hi N possible_keys: a

T

bl a r fe

key: a key_len: 10 ref: const,const rows: 1 Extra: Using where; Using index 1 row in set (#.## sec)

C-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

7.

Appendix C: Quiz Solutions

SELECT a FROM t1 WHERE a = 10 AND c = 17 mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 AND c = 17\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: ref possible_keys: a key: a key_len: 5 ref: const rows: 1 Extra: Using where; Using index 1 row in set (#.## sec)

8.

SELECT a FROM t1 WHERE a = 10 AND d = 5 mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 AND d = 5\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: ref possible_keys: a,d key: a key_len: 5 ref: const rows: 1 Extra: Using where 1 row in set (#.## sec)

e

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n 9. SELECT a FROM t1 WHEREtraa= 1 AND tbh = 1 AND c = 5 e ฺ us oanFROM o mysql> EXPLAIN SELECT t1 WHERE a = 1 AND b = 1 AND c = 5\G o t (m nse 1. row *************************** *************************** n id: 1 a r SIMPLE ce T i select_type: l t t1 uye table: type: ref g hi N possible_keys: a

T

bl a r fe

key: a key_len: 15 ref: const,const,const rows: 1 Extra: Using where; Using index 1 row in set (#.## sec)

C-3

MySQL Developer Techniques

Appendix C: Quiz Solutions

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

10. SELECT a FROM t1 WHERE a = 1 AND b = 1 AND c = 5 AND d = 1 mysql> EXPLAIN SELECT a FROM t1 WHERE a = 1 AND b = 1 AND c = 5 AND d = 1\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: ref possible_keys: a,d key: a key_len: 15 ref: const,const,const rows: 1 Extra: Using where 1 row in set (#.## sec)

11. SELECT a FROM t1 WHERE a = 10 GROUP BY b mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 GROUP BY b\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: ref possible_keys: a key: a key_len: 5 ref: const rows: 1 Extra: Using where; Using index 1 row in set (#.## sec)

e

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n a = 10thGROUP a 12. SELECT MAX(c) FROM t1 tWHERE BY b r e ฺ s n u t1 WHERE a = 10 GROUP BY b\G oMAX(c)toFROM o mysql> EXPLAIN SELECT m *************************** n 1( ense 1. row *************************** id: a r T select_type: SIMPLE lic t e table: t1 uy type: ref g hi N possible_keys: a

T

bl a r fe

key: a key_len: 5 ref: const rows: 1 Extra: Using where; Using index 1 row in set (#.## sec)

C-4

MySQL Developer Techniques

Appendix C: Quiz Solutions

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

13. SELECT a FROM t1 WHERE a = 10 AND d = 5 ORDER BY b mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 AND d = 5 ORDER BY b\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: ref possible_keys: a,d key: a key_len: 5 ref: const rows: 1 Extra: Using where 1 row in set (#.## sec)

14. SELECT a FROM t1 WHERE a BETWEEN 1 AND 10 ORDER BY b mysql> EXPLAIN SELECT a FROM t1 WHERE a BETWEEN 1 AND 10 ORDER BY b\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t1 type: index possible_keys: a key: a key_len: 15 ref: NULL rows: 1 Extra: Using where; Using index; Using filesort 1 row in set (#.## sec)

e

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n 15. SELECT a FROM t1 WHEREtraa= 10 ANDth b < 3 ORDER BY c e ฺ us oanFROM o mysql> EXPLAIN SELECT t1 WHERE a = 10 AND b < 3 ORDER BY c\G o t (m nse 1. row *************************** *************************** n id: 1 a r SIMPLE ce T i select_type: l t t1 uye table: type: ref g hi N possible_keys: a

T

bl a r fe

key: a key_len: 5 ref: const rows: 10 Extra: Using where; Using index; Using filesort 1 row in set (#.## sec)

C-5

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

APPENDIX D

e

s

bl a r fe

an r t n

uy g N

no a FURTHER PRACTICE s a h n) ideฺ v ฺ SOLUTIONS om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi

e

s

o

uy g N an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e an r t n bl a r fe

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Appendix D: Further Practice Solutions Further Practice Lab 2-C

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

1.

Thi

If necessary, install a fresh copy of the world database to remove any changes that may have been made to the data or the data structures.

mysql> DROP DATABASE IF EXISTS world; mysql> CREATE DATABASE world; mysql> USE world; mysql> SOURCE C:\world.sql

The location of the world.sql file may be different than the one displayed, please change it accordingly. 2. Create three new indexes in the world.Country table: a.

A unique index named uName that indexes the Name column.

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ m the Continent u b. A composite index named Cont_Pop that indexes and Population o G c ฺ t columns together. 1 n s2 tude t p s (Continent, mysql> CREATE INDEX Cont_Pop ON Country Population); S @ s Query OK, 239 rows affected (#.## sec) i n th0 Records: 239 Duplicates:ฺt0raWarnings: e us that indexes the Capital column. onnamedtouCapital o c. A unique index (m nse n a r INDEX euCapital ON Country (Capital); mysql> CREATE c T i l t Query 239 rows affected (#.## sec) ye OK,239 Duplicates: 0 Warnings: 0 guRecords: mysql> CREATE INDEX uname ON Country (Name); Query OK, 239 rows affected (#.## sec) Records: 239 Duplicates: 0 Warnings: 0

N

3.

Create a new index in the world.City table against the CountryCode column.

mysql> CREATE INDEX uCountryCode ON City (CountryCode); Query OK, 4079 rows affected (#.## sec) Records: 4079 Duplicates: 0 Warnings: 0

D-1

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

4.

Thi

Appendix D: Further Practice Solutions

Determine which indexes (if any) the MySQL query optimizer would consider using in the following SQL statement:

mysql> EXPLAIN SELECT DISTINCT Country.Region FROM Country -> INNER JOIN CountryLanguage ON CountryLanguage.CountryCode = Country.Code -> WHERE Language = 'Spanish'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: Country type: ALL possible_keys: PRIMARY key: NULL key_len: NULL ref: NULL rows: 239 Extra: Using temporary *************************** 2. row *************************** id: 1 select_type: SIMPLE table: CountryLanguage type: eq_ref possible_keys: PRIMARY,CountryCode key: PRIMARY key_len: 33 ref: world.Country.Code,const rows: 1 Extra: Using where; Using index; Distinct 2 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

D-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

5.

Appendix D: Further Practice Solutions

Determine which indexes (if any) the MySQL query optimizer would consider using in the following SQL statement:

mysql> EXPLAIN SELECT Name, Continent FROM Country WHERE Population > 200000000\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: Country type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 239 Extra: Using where 1 row in set (#.## sec)

e

bl a r fe

Modifying the above SQL statement, determine which indexes (if any) the MySQL query optimizer would consider using now:

ns a r t -200000000\G mysql> EXPLAIN SELECT DISTINCT Continent FROM Country WHERE Population > n o *************************** 1. row *************************** an id: 1 s select_type: SIMPLE a h table: Country ) n ideฺ v type: index ฺ possible_keys: NULL om t Gu c ฺ key: Cont_Pop 21 den key_len: 5 s t ref: NULL sp s Stu rows: 239 @ n indexthi Extra: Using where;ra Using t 1 row in set (#.## sec)nฺ se u o o to m ( e an icens r T l t e y u

g

N Thi

D-3

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

6.

Thi

Appendix D: Further Practice Solutions

Determine which indexes (if any) the MySQL query optimizer would consider using in the following SQL statement:

mysql> EXPLAIN SELECT CountryCode, MAX(Population) FROM City GROUP BY CountryCode\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4079 Extra: Using temporary; Using filesort 1 row in set (#.## sec)

e

bl a r fe

Modifying the above SQL statement, determine which indexes (if any) the MySQL query optimizer would consider using now:

s

mysql> EXPLAIN SELECT CountryCode, MAX(Population) FROM City -> WHERE CountryCode LIKE 'C%' GROUP BY CountryCode\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: City type: range possible_keys: CountryCode,uCountryCode key: CountryCode key_len: 3 ref: NULL rows: 538 Extra: Using where 1 row in set (#.## sec)

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

D-4

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Further Practice 3-C 1.

Install a copy of the sakila database into a database with the same name.

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

mysql> DROP DATABASE IF EXISTS sakila;

Thi

mysql> CREATE DATABASE sakila; mysql> USE sakila; mysql> SOURCE C:\sakila.sql

2.

The location of the sakila.sql file may different than the one displayed, please change it accordingly. Using the sakila database, review the structure of the customer, film, inventory and rental tables.

s

an r t n

mysql> SHOW COLUMNS FROM customer; +-------------+------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+------------------+------+-----+---------+----------------+ | customer_id | int(10) unsigned | NO | PRI | NULL | auto_increment | | store_id | int(10) unsigned | NO | MUL | | | | last_name | varchar(45) | NO | | | | | first_name | varchar(45) | NO | | | | | address | varchar(255) | NO | | | | | address2 | varchar(255) | YES | | NULL | | | city | varchar(60) | NO | | | | | district | varchar(60) | NO | | | | | country | varchar(60) | NO | | | | | phone | varchar(50) | NO | | | | | postal_code | varchar(20) | NO | | | | | active | tinyint(1) | NO | | 1 | | +-------------+------------------+------+-----+---------+----------------+ 12 rows in set (#.## sec)

e

bl a r fe

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

y SHOW COLUMNS FROM film; umysql> g +------------------+---------------------+------+-----+---------+----------------+ N | Field | Type | Null | Key | Default | Extra | +------------------+---------------------+------+-----+---------+----------------+ | film_id | int(10) unsigned | NO | PRI | NULL | auto_increment | | category_id | int(10) unsigned | NO | | | | | title | varchar(255) | NO | MUL | | | | description | text | YES | | NULL | | | rental_duration | tinyint(3) unsigned | NO | | 3 | | | rental_rate | decimal(4,2) | NO | | 4.99 | | | length | int(10) unsigned | YES | | NULL | | | replacement_cost | decimal(5,2) | NO | | | | | rating | enum('G','PG', ... | YES | | G | | +------------------+---------------------+------+-----+---------+----------------+ 9 rows in set (#.## sec) mysql> SHOW COLUMNS FROM inventory; +--------------+------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +--------------+------------------+------+-----+---------+----------------+ | inventory_id | int(10) unsigned | NO | PRI | NULL | auto_increment | | film_id | int(10) unsigned | NO | MUL | | | | store_id | int(10) unsigned | NO | MUL | | | +--------------+------------------+------+-----+---------+----------------+ 3 rows in set (#.## sec)

D-5

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

Appendix D: Further Practice Solutions

mysql> SHOW COLUMNS FROM rental; +--------------+------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------+------------------+------+-----+---------+-------+ | rent_date | datetime | NO | PRI | | | | inventory_id | int(10) unsigned | NO | PRI | | | | customer_id | int(10) unsigned | NO | PRI | | | | return_date | datetime | YES | | NULL | | | staff_id | int(10) unsigned | NO | MUL | | | +--------------+------------------+------+-----+---------+-------+ 5 rows in set (#.## sec)

3.

Using the rental table, determine the maximum rental days where a customer checked out a video and returned it. In addition, include the average length of days that customers (who have returned the videos) have kept the videos.

mysql> SELECT MAX(DATEDIFF(return_date,rent_date)) AS Max_Days, -> AVG(DATEDIFF(return_date,rent_date)) AS Avg_Days -> FROM rental WHERE return_date IS NOT NULL; +----------+----------+ | Max_Days | Avg_Days | +----------+----------+ | 10 | 5.0263 | +----------+----------+ 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

D-6

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

4.

Thi

Appendix D: Further Practice Solutions

Create a report that identifies videos that are currently out. Include in the report the customers name and contact info, the video that they have checked out and the number of days they have had the movie checked out (from the current date). Note: Due to this data being slightly outdated, the days checked out will be quite lengthy.

mysql> SELECT customer.first_name, customer.last_name, customer.phone, film.title, -> DATEDIFF(NOW(),rental.rent_date) AS days_checked_out -> FROM customer,film,inventory,rental -> WHERE customer.customer_id = rental.customer_id -> AND rental.inventory_id = inventory.inventory_id -> AND inventory.film_id = film.film_id -> AND return_date IS NULL LIMIT 20; +------------+------------+--------------+---------------------+------------------+ | first_name | last_name | phone | title | days_checked_out | +------------+------------+--------------+---------------------+------------------+ | BRANDON | HUEY | 99883471275 | ACE GOLDFINGER | 1160 | | CARMEN | OWENS | 272234298332 | AFFAIR PREJUDICE | 1156 | | SETH | HANNON | 864392582257 | AFRICAN EGG | 1156 | | TRACY | COLE | 371490777743 | ALI FOREVER | 1157 | | MARCIA | DEAN | 727785483194 | ALONE TRIP | 1160 | | CECIL | VINES | 879347453467 | AMADEUS HOLY | 1160 | | MARIE | TURNER | 177727722820 | AMERICAN CIRCUS | 1160 | | JOE | GILLILAND | 53912826864 | AMISTAD MIDSUMMER | 1157 | | EDWARD | BAUGH | 46568045367 | ARMAGEDDON LOST | 1161 | | BETH | FRANKLIN | 470884141195 | BAKED CLEOPATRA | 1159 | | GINA | WILLIAMSON | 584316724815 | BANG KWAI | 1158 | | MELVIN | ELLINGTON | 368284120423 | BASIC EASY | 1156 | | SONIA | GREGORY | 195003555232 | BERETS AGENT | 1162 | | FLORENCE | WOODS | 330838016880 | BLADE POLISH | 1156 | | TYLER | WREN | 211256301880 | BLANKET BEVERLY | 1160 | | MILDRED | BAILEY | 250767749542 | BOOGIE AMELIE | 1161 | | ANNA | HILL | 911872220378 | BOULEVARD MOB | 1161 | | ROBIN | HAYES | 942570536750 | BOUND CHEAPER | 1157 | | LOIS | BUTLER | 345679835036 | BUBBLE GROSSE | 1160 | | LISA | ANDERSON | 635297277345 | BULL SHAWSHANK | 1162 | ... | BECKY | MILES | 648482415405 | VIRGINIAN PLUTO | 1158 | | JENNY | CASTRO | 62781725285 | VOLCANO TEXAS | 1156 | | GREG | ROBINS | 206060652238 | WANDA CHAMBER | 1157 | | REGINA | BERRY | 201705577290 | WEDDING APOLLO | 1156 | | NAOMI | JENNINGS | 959949395183 | WILD APOLLO | 1159 | | JEREMY | HURTADO | 600264533987 | WINDOW SIDE | 1156 | | NATALIE | MEYER | 873492228462 | WOMEN DORADO | 1162 | | NEIL | RENNER | 478380208348 | WORLD LEATHERNECKS | 1158 | | LOUIS | LEONE | 45554316010 | ZHIVAGO CORE | 1162 | +------------+-------------+--------------+--------------------+------------------+ 184 rows in set (#.## sec)

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

Your output may be slightly different than the output displayed here due to the use of the NOW() function which will affect the output based on when this statement is run.

D-7

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Further Practice Lab 5-A Solution

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Using the divide and conquer technique calculate the total tax, the tax on revenue % and the total % of the salaries that the employees get to keep when we have the following list of employees:

Thi

Name

Salary

Johan

30,000

Max

55,000

Tobias

90,000

mysql> CREATE TABLE taxrate ( -> base DECIMAL(8,2) NOT NULL, -> threshold DECIMAL (10,2) NOT NULL, -> tax DECIMAL (3,2) NOT NULL -> );

e

bl a r fe

s

mysql> INSERT INTO taxrate VALUES -> (0, 5687, 0), -> (5687, 11344, 0.055), -> (11344, 25195, 0.14), -> (25195, 67546, 0.3), -> (67546, 9999999, 0.4);

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu mysql> CREATE TABLE salaries ( @ i n -> name char(10) NOT NULL, h a t r t -> salary DECIMAL(10,2) NOT NULL e -> ); onฺ to us o (m nse n a r e csalaries T i l t mysql> INSERT INTO e-> ("Johan", 30000), VALUES y u g -> ("Max", 55000),

N

->

an r t n

("Tobias", 90000);

mysql> CREATE VIEW v_salaries AS -> SELECT *, salary*0.815 AS revenue FROM salaries;

D-8

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Developer Techniques

Thi

mysql> SELECT name, salary, taxpart as totaltax, -> ROUND(taxpart/revenue*100,2) AS taxpct, -> ROUND((revenue - taxpart)/salary*100,2) AS totalpct -> FROM ( -> SELECT name, salary, revenue, -> ROUND( -> SUM( -> IF(revenue 0, -> IF(revenue < threshold, -> (revenue - base) * tax, -> (threshold - base) * tax -> ) -> ) -> ), 2) AS taxpart -> FROM v_salaries CROSS JOIN taxrate GROUP BY name -> ) AS tmp; +--------+----------+----------+--------+----------+ | name | salary | totaltax | taxpct | totalpct | +--------+----------+----------+--------+----------+ | Johan | 30000.00 | 2174.26 | 8.89 | 74.25 | | Max | 55000.00 | 8167.56 | 18.22 | 66.65 | | Tobias | 90000.00 | 17305.46 | 23.59 | 62.27 | +--------+----------+----------+--------+----------+

Appendix D: Further Practice Solutions

e

bl a r fe

s

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

uy g N

D-9

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Further Practice Lab 8-A Step 1: Examine the indexes already present in the Country table

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Utilizing the world database, list the indexes currently in place for the Country table: mysql> SHOW INDEX FROM Country\G *************************** 1. row *************************** Table: Country Non_unique: 0 Key_name: PRIMARY Seq_in_index: 1 Column_name: Code Collation: A Cardinality: 239 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: 1 row in set (#.## sec)

e

bl a r fe

s

an r t n

no a sconcerning each of the a The SHOW INDEX command will produce a large amount of information h ฺ on the Code (the indexes associated with the table requested. In this case, the index n)listediisdbased e v ฺ three digit country code associated with the country) column. om t Gu c ฺ 1 in theenCountry table 2column Step 2: Create and test an index on the Name s t p Stud s @ Create an index on the Name column of the Country is table: n h a t r ฺt Country se ADD INDEX (Name); n u mysql> ALTERoTABLE o239 rowstoaffected (#.## sec) Query OK, m ( e sDuplicates: n 239 Records: 0 Warnings: 0 n a r e c T li t e y u

g

N Thi

D-10

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Obtain information about how MySQL would execute the following SELECT statement:

Thi

mysql> SELECT Name, Population FROM Country WHERE Name LIKE '%land'; mysql> EXPLAIN SELECT Name, Population FROM Country -> WHERE Name LIKE '%land'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: Country type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 239 Extra: Using where 1 row in set (#.## sec)

s

an r t n

Obtain information about how MySQL would execute the following SELECT statement:

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

e

bl a r fe

mysql> SELECT Name, Population FROM Country WHERE Name LIKE 'S%'; mysql> EXPLAIN SELECT Name, Population -> FROM Country WHERE Name LIKE 'S%'\G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: Country type: range possible_keys: Name key: Name key_len: 52 ref: NULL rows: 32 Extra: Using where 1 row in set (#.## sec)

uy Which SELECT statement would perform better? Why? g N

__________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ In this first SELECT statement, the index was blocked from being used (due to the leading %) and a full table scan would need to be performed (all 239 rows). In the second SELECT statement, the server would only need to evaluate 32 rows (versus having to search all 239 rows of the table) using the WHERE clause. This is exactly what was to be expected from the search due to the fact that we were utilizing an indexed column and searching from the left most portion of the field.

D-11

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Step 3: Create and update a reverse column based on the Name column of the world database

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Add a new indexed column to the Country table called NameReverse that allows for 52 characters and is located after the Name column. mysql> ALTER TABLE Country ADD NameReverse CHAR(52) AFTER Name, ADD -> INDEX (NameReverse); Query OK, 239 rows affected (#.## sec) Records: 239 Duplicates: 0 Warnings: 0 This command added a new column (NameReverse) to the Country table (right after the Name column) and then followed up adding an index on that same column. Fill this new column with a mirror image of the Name column using the REVERSE function on all the current records in the table. mysql> UPDATE Country SET NameReverse = REVERSE(Name); Query OK, 239 rows affected (#.## sec) Rows matched: 239 Changed: 239 Warnings: 0

ns a r t - of the This UPDATE command accomplished the task of updating each record with a mirror n image o Name column of the Country table. an s a h Step 4: Search for a string at the "end" of a column ) n ideฺ v ฺ m SELECT u statement (similar to the Obtain information about how MySQL would execute theofollowing G c ฺ t poor performing SELECT statement presented earlier): 21 den s t tuCountry mysql> SELECT Name, Population sp sFROM S @ -> WHERE NameReverse LIKE i 'dnal%'; an Name, thPopulation r mysql> EXPLAIN ฺSELECT t e s LIKE 'dnal%'\GFROM Country n -> WHERE NameReverse u o o to *************************** 1. row *************************** m ( e s id: 1 n a icen SIMPLE r select_type: T ltable: Country t e y u

g

N Thi

e

bl a r fe

type: range possible_keys: NameReverse key: NameReverse key_len: 53 ref: NULL rows: 14 Extra: Using where 1 row in set (#.## sec)

The server is now able to utilize the index due the lead % being removed and the result is 14 rows having to be evaluated versus a fill table scan (293 rows).

D-12

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Step 5: Create triggers to support long-term maintenance of the new column

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

After the NameReverse column has been added to the Country table, keeping this column updated is a job for triggers. Create both an update trigger (to catch changes to e-mail addresses) and an insert trigger (for new data being added to the table).

Thi

mysql> CREATE TRIGGER Country_BU_NameReverse BEFORE UPDATE ON Country -> FOR EACH ROW BEGIN SET NEW.NameReverse = REVERSE(NEW.Name); Query OK, 0 rows affected (#.## sec) mysql> CREATE TRIGGER Country_BI_NameReverse BEFORE INSERT ON Country -> FOR EACH ROW BEGIN SET NEW.NameReverse = REVERSE(NEW.Name); Query OK, 0 rows affected (#.## sec)

Step 6: Test the alteration and addition of new rows With the triggers in place to handle both updates and additions to the Country table, it is best to test it to make sure that the triggers will work as expected. Test the alteration of existing data with the following SQL statements:

e

bl a r fe

s

an r t n

no a s a h ฺ LIKE 'dnal%'; n) NameReverse mysql> SELECT Name, NameReverse FROM Countryฺv WHERE e d i +------------------+------------------+ m o t Gu c | Name | NameReverse | ฺ 1 n +------------------+------------------+ 2 e s d t | New Zealand | dnalaeZ p s weN s St||u | Iceland | dnalecI @ i | n | Beigeland |adnalegieB h t r t | Ireland | dnalerI | e ฺ | dnaliahT s n u | Thailand | o o o t ... m se | dnalsI samtsirhC | | Christmas Island n ( Island n a r e | Bouvet | dnalsI tevuoB | lic tT mysql> UPDATE Country SET Name='Beigeland' WHERE Name='Greenland'; Query OK, 1 row affected (#.## sec) Rows matched: 1 Changed: 1 Warnings: 0

N

e guy

+------------------+------------------+ 12 rows in set (#.## sec)

The first SQL statement updated the Country table by changing the name of 'Greenland' to 'Beigeland'. The second SQL statement retrieved all the records that ended in land to see if the change was also passed onto the NameReverse column. From the example above, the name column that was altered also changes the NameReverse column from 'dnalneerG' to 'dnaleqieB'; thus demonstrating that the trigger for updating records worked.

D-13

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Now test the triggers that manage new records being inserted by entering the following SQL statements into the mysql client:

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

mysql> INSERT INTO Country (Code, Name) VALUES ('SUN','Sunland'); Query OK, 1 row affected (#.## sec) mysql> SELECT Name, NameReverse FROM Country WHERE NameReverse LIKE 'dnal%'; +------------------+------------------+ | Name | NameReverse | +------------------+------------------+ | New Zealand | dnalaeZ weN | | Iceland | dnalecI | | Beigeland | dnalegieB | | Ireland | dnalerI | | Thailand | dnaliahT | | Swaziland | dnalizawS | | Finland | dnalniF | | Sunland | dnalnuS | | Poland | dnaloP | | Switzerland | dnalreztiwS | | Norfolk Island | dnalsI klofroN | | Christmas Island | dnalsI samtsirhC | | Bouvet Island | dnalsI tevuoB | +------------------+------------------+ 13 rows in set (#.## sec)

s

an r t n

no a s a h ) by adding ntable The first SQL statement inserted a new record into the Country eฺ the country code v ฺ d i u in that order.. The (Code) and country name (Name) with the values 'SUN' 'Sunland' G coinmlandand ฺ t second SQL statement retrieved all the records that1 ended to see if the insert took place and the n 2 e s new country name was also passed onto thetNameReverse column. From the example above, the d u p t s column that was inserted also added the mirror image S ('dnalnuS') of the Name column to the @ s i n NameReverse column; thus demonstrating that the trigger for inserting records worked also. h a et r t ฺ on to us o (m nse n a r ce T i l t uye

g

N Thi

e

bl a r fe

D-14

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Thi



e

s

bl a r fe

an r t n

uy g N

no a asRecovery h Better Performance, Scalability, )and n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Server 5.5

Thi

• • • • • •

InnoDB default storage engine InnoDB performance improvements MySQL performance improvements Improved availability Improved usability Improved scalability

e

s

bl a r fe

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t tu sp InnoDB S MySQL y Q 5.5 introduces a re-architected that includes many y pperformance @ s i n h a t r and scalability featuresnin ฺt addition seto what is available in the previous versions. u o to MySQL server performance and scalability across MySQL Server(m 5.5ooptimizes e ns specifically on multi-CPU/core, hyper-threaded heterogeneous an platforms, r e c T li t architecture. e y gu CONFIDENTIAL – HIGHLY RESTRICTED

N

MySQL Developer Techniques Appendix E - 2

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

InnoDB: Default Storage Engine

Thi

From MySQL 5.5, InnoDB is the default storage engine. • ACID Transactions • FKs • Crash Recovery

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t tu sp s ACID-compliant S Oracle’s InnoDB pprovides highly g y efficient p transactional @ i n h a t r capabilities, and includes elements that assure high e ฺt unique sarchitectural n u o o performance and scalability. toThis storage engine is structurally designed to handle m ( e transactional ns that require crash recovery, referential integrity, high an applications r e c T levels and SLA exceeding response times. li t of user concurrency, e y gu CONFIDENTIAL – HIGHLY RESTRICTED

N

MySQL Developer Techniques Appendix E - 3

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

InnoDB Performance Improvements

Thi

• • • •

Multiple buffer pool instances Multiple rollback segments Extended change buffering Improved recovery times

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t tu sp issupdating S • No more waitingg when a thread p g the buffer pool. p @ i n h a t r ฺtsegments • Multiple rollback se– now support of 128K concurrent transactions n u o o to delete buffering and purge buffering. • Change(m buffering includes e n ens a r T lic 5.5, there are default optimizations designed to speed up the e•t From MySQL CONFIDENTIAL – HIGHLY RESTRICTED

uy g N

scanning and applying of redo logs, so the next restart is faster. Users who previously sized redo logs artificially low because of slow recovery times can now increase the log file size without concern.

MySQL Developer Techniques Appendix E - 4

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL Performance Improvements

Thi

• • • • •

Better metadata locking within transactions Split LOCK_open mutex Eliminated LOCK_alarm mutex as bottleneck Eliminated LOCK_thread_count as bottleneck Improved performance/scale on Win32, 64

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t p Stu sthat Transaction serialization avoids one @ is session pperforms a data definition n h a t r language (DDL) statement e that is used in an incomplete transaction in ฺt onuastable n o o another session. to m ( e s for speeding up and scaling up Windows-specific an icenfeatures r T l API calls for much of the I/O done inside MySQL t Windows e • y gu CONFIDENTIAL – HIGHLY RESTRICTED

N

•

Ability to build engines and other plug-ins as DLLs on Windows

•

Network support for autodetecting the MAC address

•

Much cleanup and simplifying of threading code

MySQL Developer Techniques Appendix E - 5

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Improved Availability

Thi

• • • •

Semi-synchronous replication Replication heartbeat Replication slave fsync options Automatic relay log recovery

e

s

bl a r fe

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t p Stu sbeen From MySQL y Q 5.5,, replication p has @ isenhanced to: n h a t r ฺt ubetween • Ensure data consistency se master and slave n o o tifo replication is not working m ( • Immediately detect e n ens a r c T •t Allow alicrashed slave to automatically recover from the master relay log e y gu • Allow users to filter events for specific servers CONFIDENTIAL – HIGHLY RESTRICTED

N

•

Correctly convert data types between masters and slaves

MySQL Developer Techniques Appendix E - 6

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Improved Usability • • • • • • •

SIGNAL/RESIGNAL More partitioning options Replication server filtering Replication slave side type conversions Individual log flushing Pluggable external authentication, audit interfaces PERFORMANCE_SCHEMA

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t sp ysofSthetustored objects • MySQL y Q 5.5 extends the usability j and table/index @ i n h a t r partitioning features. ฺt use n o o RESIGNAL to statements allow you to implement exception• SIGNAL and m ( e nsin your stored procedures, stored functions, triggers, events, handling an iclogic r e T l applications. t and database e y u CONFIDENTIAL – HIGHLY RESTRICTED

g

N Thi

•

With the RANGE COLUMNS and LIST COLUMNS clauses of the CREATE TABLE statement, statement partitioning is more flexible and can optimize queries better.

•

Performance schema provides tables that let you see performance data in real time, or historically.

MySQL Developer Techniques Appendix E - 7

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

Improved Scalability

Thi

• Scales on multi-core with InnoDB 1.0 and even further with InnoDB built-in • Improved log sys mutex • Separate flush list mutex • Improved purge scheduling

e

bl a r fe

s

an r t n

no a s a h n) ideฺ v ฺ om t Gu c ฺ 21 den s t tu sp stoSincrease • Transactions pper second continue beyond y 16 coresThe internal @ i n h a t r processing happens e waiting and less blocking of other operations ฺt withusless n o o buffertopool. You do not need to do any additional. involving the m ( sliste mutex means reduced contention with buffer pool n flush n • Separate a r e lic and faster InnoDB. You do not need to do any additional. et Toperations CONFIDENTIAL – HIGHLY RESTRICTED

uy g N

•

From MySQL 5.5, purge can run in its own thread, thereby permitting more concurrency. concurrency

•

Generally, improved performance levels at higher transactions per second and user connection loads, so applications remain responsive even when the physical server resources are saturated. (Mutexes are in-memory structures that prevent different threads from interfering with each others’ changes to important memory areas such as the buffer pool.)

MySQL Developer Techniques Appendix E - 8

MySQL 5.5 Scales on Multi Core

Thi

MySQL 5.5.4 MySQL 5.5.3

Transactions/S Second

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

S B SysBench h Read R d Write W it

e

MySQL 5.1

an r t n

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e CONFIDENTIAL – HIGHLY RESTRICTED

AMD Opteron 7160 (Magny-Cours) @2100 MHz 64 GB memory 2 x Intel X25E SSD drives OS is Oracle Enterprise Linux with the Enterprise Kernel 4 sockets with a total of 48 cores.

uy g N

MySQL Developer Techniques Appendix E - 9

s

bl a r fe

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL 5.5 SysBench Benchmarks Linux

Thi

MySQL 5.5.6 (New InnoDB)

MySQL 5.1.50 (InnoDB Plug-in)

e

bl a r fe

MySQL 5.1.50 (InnoDB built-in)

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Intel Xeon X7460 x86_64 4 CPU x 6 Cores/CPU 2.66 GHz, 32GB RAM Fedora 10

CONFIDENTIAL – HIGHLY RESTRICTED

uy g N

MySQL Developer Techniques Appendix E - 10

s

an r t n

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL 5.5 SysBench Benchmarks Linux

Thi

MySQL 5.5.6 (New InnoDB)

MySQL 5.1.50 (InnoDB Plug-in)

e

bl a r fe

MySQL 5.1.50 (InnoDB built-in)

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e

Intel Xeon X7460 x86_64 4 CPU x 6 Cores/CPU 2.66 GHz, 32GB RAM Fedora 10

CONFIDENTIAL – HIGHLY RESTRICTED

uy g N

MySQL Developer Techniques Appendix E - 11

s

an r t n

MySQL 5.5 Scales on Multi Core

Thi

Sysbench OLTP: Read / Write Performance

Transactions Per Second

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

S B SysBench h Read R d Write W it

10000 8000

e

6000

bl a r fe

InnoDB

4000

MyISAM

2000 0

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e 6

12

18

24

CPU Cores

CONFIDENTIAL – HIGHLY RESTRICTED

30

36

AMD Opteron 7160 (Magny-Cours) @2100 MHz 64 GB memory 2 x Intel X25E SSD drives OS is Oracle Enterprise Linux with the Enterprise Kernel 4 sockets with a total of 48 cores.

uy g N

MySQL Developer Techniques Appendix E - 12

s

an r t n

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL 5.5 SysBench Benchmarks Windows

Thi

MySQL 5.5.6 (New InnoDB)

MySQL 5.1.50 (InnoDB Plug-in)

e

bl a r fe

MySQL 5.1.50 (InnoDB built-in)

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e Intel x86 x86_64 64

s

an r t n

4 CPU x 2 Cores/CPU 3.166 GHz, 8GB RAM Windows Server 2008

CONFIDENTIAL – HIGHLY RESTRICTED

uy g N

MySQL Developer Techniques Appendix E - 13

13

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2016, Oracle and/or its affiliatesฺ

MySQL 5.5 SysBench Benchmarks Windows

Thi

MySQL 5.5.6 (New InnoDB)

MySQL 5.1.50 (InnoDB Plug-in)

e

bl a r fe

MySQL 5.1.50 (InnoDB built-in)

o

an s a h ) n ideฺ v ฺ om t Gu c ฺ 21 den s t sp s Stu @ i n h a t r t e onฺ to us o (m nse n a r ce T i l t e Intel x86 x86_64 64

s

an r t n

4 CPU x 2 Cores/CPU 3.166 GHz, 8GB RAM Windows Server 2008

CONFIDENTIAL – HIGHLY RESTRICTED

uy g N

MySQL Developer Techniques Appendix E - 14

14