Python
March 16, 2017 | Author: Edith Soto | Category: N/A
Short Description
Download Python...
Description
www.it-ebooks.info
Learning Python
www.it-ebooks.info
www.it-ebooks.info
FOURTH EDITION
Learning Python
Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo
www.it-ebooks.info
Learning Python, Fourth Edition
Editor: Production Editor: Copyeditor: Production Services: Printing History:
Indexer: Cover Designer: Interior Designer: Illustrator:
www.it-ebooks.info
www.it-ebooks.info
www.it-ebooks.info
Table of Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxi
Part I. Getting Started 1. A Python Q&A Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
vii
www.it-ebooks.info
2. How Python Runs Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3. How You Run Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
viii | Table of Contents
www.it-ebooks.info
Part II. Types and Operations 4. Introducing Python Object Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Table of Contents | ix
www.it-ebooks.info
5. Numeric Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6. The Dynamic Typing Interlude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
x | Table of Contents
www.it-ebooks.info
7. Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
8. Lists and Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Table of Contents | xi
www.it-ebooks.info
9. Tuples, Files, and Everything Else . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Part III. Statements and Syntax 10. Introducing Python Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
xii | Table of Contents
www.it-ebooks.info
11. Assignments, Expressions, and Prints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
12. if Tests and Syntax Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
Table of Contents | xiii
www.it-ebooks.info
13. while and for Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
14. Iterations and Comprehensions, Part 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
xiv | Table of Contents
www.it-ebooks.info
15. The Documentation Interlude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
Part IV. Functions 16. Function Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
Table of Contents | xv
www.it-ebooks.info
17. Scopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
18. Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
xvi | Table of Contents
www.it-ebooks.info
19. Advanced Function Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
20. Iterations and Comprehensions, Part 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
Table of Contents | xvii
www.it-ebooks.info
Part V. Modules 21. Modules: The Big Picture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
22. Module Coding Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
xviii | Table of Contents
www.it-ebooks.info
23. Module Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561
24. Advanced Module Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583
Table of Contents | xix
www.it-ebooks.info
Part VI. Classes and OOP 25. OOP: The Big Picture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611
26. Class Coding Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625
xx | Table of Contents
www.it-ebooks.info
27. A More Realistic Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643
28. Class Coding Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681
Table of Contents | xxi
www.it-ebooks.info
29. Operator Overloading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705
xxii | Table of Contents
www.it-ebooks.info
30. Designing with Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737
31. Advanced Class Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 773
Table of Contents | xxiii
www.it-ebooks.info
Part VII. Exceptions and Tools 32. Exception Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825
33. Exception Coding Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835
xxiv | Table of Contents
www.it-ebooks.info
34. Exception Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 857
35. Designing with Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 873
Table of Contents | xxv
www.it-ebooks.info
Part VIII. Advanced Topics 36. Unicode and Byte Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895
xxvi | Table of Contents
www.it-ebooks.info
37. Managed Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 941
Table of Contents | xxvii
www.it-ebooks.info
38. Decorators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 983
xxviii | Table of Contents
www.it-ebooks.info
39. Metaclasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1051
Part IX. Appendixes A. Installation and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1089 B. Solutions to End-of-Part Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1101 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1139
Table of Contents | xxix
www.it-ebooks.info
www.it-ebooks.info
Preface
xxxi
www.it-ebooks.info
About This Fourth Edition
print
Coverage for Both 3.0 and 2.6
print nonlocal
xxxii | Preface
print format
www.it-ebooks.info
format with
New Chapters
Preface | xxxiii
www.it-ebooks.info
Changes to Existing Material
map
zip __contains__ __bool__
format exec
xxxiv | Preface
__index__
www.it-ebooks.info
Specific Language Extensions in 2.6 and 3.0
list
Extension
Covered in chapter(s)
The print function in 3.0
11
The nonlocal x,y statement in 3.0
17
The str.format method in 2.6 and 3.0
7
String types in 3.0: str for Unicode text, bytes for binary data
7, 36
Text and binary file distinctions in 3.0
9, 36
Class decorators in 2.6 and 3.0: @private('age')
31, 38
New iterators in 3.0: range, map, zip
14, 20
Dictionary views in 3.0: D.keys, D.values, D.items
8, 14
Division operators in 3.0: remainders, / and //
5
Set literals in 3.0: {a, b, c}
5
Set comprehensions in 3.0: {x**2 for x in seq}
4, 5, 14, 20
Dictionary comprehensions in 3.0: {x: x**2 for x in seq}
4, 8, 14, 20
Binary digit-string support in 2.6 and 3.0: 0b0101, bin(I)
5
The fraction number type in 2.6 and 3.0: Fraction(1, 3)
5
Function annotations in 3.0: def f(a:99, b:str)->int
19
Keyword-only arguments in 3.0: def f(a, *b, c, **d)
18, 20
Extended sequence unpacking in 3.0: a, *b = seq
11, 13
Relative import syntax for packages enabled in 3.0: from .
23
Context managers enabled in 2.6 and 3.0: with/as
33, 35
Exception syntax changes in 3.0: raise, except/as, superclass
33, 34
Preface | xxxv
www.it-ebooks.info
Extension
Covered in chapter(s)
Exception chaining in 3.0: raise e2 from e1
33
Reserved word changes in 2.6 and 3.0
11
New-style class cutover in 3.0
31
Property decorators in 2.6 and 3.0: @property
37
Descriptor use in 2.6 and 3.0
31, 38
Metaclass use in 2.6 and 3.0
31, 39
Abstract base classes support in 2.6 and 3.0
28
Specific Language Removals in 3.0
Removed
Replacement
Covered in chapter(s)
reload(M)
imp.reload(M) (or exec)
3, 22
apply(f, ps, ks)
f(*ps, **ks)
18
`X`
repr(X)
5
X Y
X != Y
5
long
int
5
9999L
9999
5
D.has_key(K)
K in D (or D.get(key) != None)
8
raw_input
input
3, 10
old input
eval(input())
3
xrange
range
14
file
open (and io module classes)
9
X.next
X.__next__, called by next(X)
14, 20, 29
X.__getslice__
X.__getitem__ passed a slice object
7, 29
X.__setslice__
X.__setitem__ passed a slice object
7, 29
reduce
functools.reduce (or loop code)
14, 19
execfile(filename)
exec(open(filename).read())
3
exec open(filename)
exec(open(filename).read())
3
0777
0o777
5
print x, y
print(x, y)
11
xxxvi | Preface
www.it-ebooks.info
Removed
Replacement
Covered in chapter(s)
print >> F, x, y
print(x, y, file=F)
11
print x, y,
print(x, y, end=' ')
11
u'ccc'
'ccc'
7, 36
'bbb' for byte strings
b'bbb'
7, 9, 36
raise E, V
raise E(V)
32, 33, 34
except E, X:
except E as X:
32, 33, 34
def f((a, b)):
def f(x): (a, b) = x
11, 18, 20
file.xreadlines
for line in file: (or X=iter(file))
13, 14
D.keys(), etc. as lists
list(D.keys()) (dictionary views)
8, 14
map(), range(), etc. as lists
list(map()), list(range()) (built-ins)
14
map(None, ...)
zip (or manual code to pad results)
13, 20
X=D.keys(); X.sort()
sorted(D) (or list(D.keys()))
4, 8, 14
cmp(x, y)
(x > y) - (x < y)
29
X.__cmp__(y)
__lt__, __gt__, __eq__, etc.
29
X.__nonzero__
X.__bool__
29
X.__hex__, X.__oct__
X._index__
29
Sort comparison functions
Use key=transform or reverse=True
8
Dictionary , =
Compare sorted(D.items()) (or loop code)
8, 9
types.ListType
list (types is for nonbuilt-in names only)
9
__metaclass__ = M
class C(metaclass=M):
28, 31, 39
__builtin__
builtins (renamed)
17
Tkinter
tkinter (renamed)
18, 19, 24, 29, 30
sys.exc_type, exc_value
sys.exc_info()[0], [1]
34, 35
function.func_code
function.__code__
19, 38
__getattr__ run by built-ins
Redefine __X__ methods in wrapper classes
30, 37, 38
-t, –tt command-line switches
Inconsistent tabs/spaces use is always an error
10, 12
from ... *, within a function
May only appear at the top level of a file
22
import mod, in same package
from . import mod, package-relative form
23
class MyException:
class MyException(Exception):
34
exceptions module
Built-in scope, library manual
34
thread, Queue modules
_thread, queue (both renamed)
17
anydbm module
dbm (renamed)
27
cPickle module
_pickle (renamed, used automatically)
9
os.popen2/3/4
subprocess.Popen (os.popen retained)
14
String-based exceptions
Class-based exceptions (also required in 2.6)
32, 33, 34
Preface | xxxvii
www.it-ebooks.info
Removed
Replacement
Covered in chapter(s)
String module functions
String object methods
7
Unbound methods
Functions (staticmethod to call via instance)
30, 31
Mixed type comparisons, sorts
Nonnumeric mixed type comparisons are errors
5, 9
About The Third Edition
The Third Edition’s Python Language Changes
xxxviii | Preface
www.it-ebooks.info
B if A else C with as try except finally
sorted sum any all enumerate
distutils unittest
doctest True
False
sys.exc_info apply
reduce
apply
The Third Edition’s Python Training Changes
for
Preface | xxxix
www.it-ebooks.info
The Third Edition’s Structural Changes
The Third Edition’s Scope Changes
xl | Preface
www.it-ebooks.info
About This Book
This Book’s Prerequisites
This Book’s Scope and Other Books
Preface | xli
www.it-ebooks.info
xlii | Preface
www.it-ebooks.info
This Book’s Style and Structure
Preface | xliii
www.it-ebooks.info
__name__
xliv | Preface
www.it-ebooks.info
Book Updates ^H^H^H
About the Programs in This Book
Using Code Examples
Preface | xlv
www.it-ebooks.info
Font Conventions
Constant width
Constant width bold
Constant width italic
xlvi | Preface
www.it-ebooks.info
% C:\Python30> % >>> ...
#
Safari® Books Online
How to Contact Us
Preface | xlvii
www.it-ebooks.info
Acknowledgments
xlviii | Preface
www.it-ebooks.info
Preface | xlix
www.it-ebooks.info
www.it-ebooks.info
PART I
Getting Started
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 1
A Python Q&A Session
Why Do People Use Python?
3
www.it-ebooks.info
Software Quality
4 | Chapter 1: A Python Q&A Session
www.it-ebooks.info
Developer Productivity
Is Python a “Scripting Language”?
import this
Is Python a “Scripting Language”? | 5
www.it-ebooks.info
6 | Chapter 1: A Python Q&A Session
www.it-ebooks.info
OK, but What’s the Downside?
Who Uses Python Today?
Who Uses Python Today? | 7
www.it-ebooks.info
8 | Chapter 1: A Python Q&A Session
www.it-ebooks.info
What Can I Do with Python?
Systems Programming
GUIs
What Can I Do with Python? | 9
www.it-ebooks.info
Internet Scripting
Component Integration
10 | Chapter 1: A Python Q&A Session
www.it-ebooks.info
Database Programming
pickle
Rapid Prototyping
Numeric and Scientific Programming
What Can I Do with Python? | 11
www.it-ebooks.info
Gaming, Images, Serial Ports, XML, Robots, and More
xml
How Is Python Supported?
12 | Chapter 1: A Python Q&A Session
xmlrpclib
www.it-ebooks.info
What Are Python’s Technical Strengths?
It’s Object-Oriented
It’s Free
What Are Python’s Technical Strengths? | 13
www.it-ebooks.info
It’s Portable
14 | Chapter 1: A Python Q&A Session
www.it-ebooks.info
It’s Powerful
What Are Python’s Technical Strengths? | 15
www.it-ebooks.info
It’s Mixable
It’s Easy to Use
16 | Chapter 1: A Python Q&A Session
www.it-ebooks.info
It’s Easy to Learn
It’s Named After Monty Python
How Does Python Stack Up to Language X?
How Does Python Stack Up to Language X? | 17
www.it-ebooks.info
Chapter Summary
18 | Chapter 1: A Python Q&A Session
www.it-ebooks.info
Test Your Knowledge: Quiz
import this
Test Your Knowledge: Answers
Test Your Knowledge: Answers | 19
www.it-ebooks.info
import this
Python Is Engineering, Not Art
20 | Chapter 1: A Python Q&A Session
www.it-ebooks.info
Test Your Knowledge: Answers | 21
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 2
How Python Runs Programs
Introducing the Python Interpreter
23
www.it-ebooks.info
Program Execution
The Programmer’s View
print('hello world') print(2 ** 100)
print
print
24 | Chapter 2: How Python Runs Programs
www.it-ebooks.info
Program Execution | 25
www.it-ebooks.info
print hello world 1267650600228229401496703205376
C:\temp> python script0.py hello world 1267650600228229401496703205376
Python’s View
Byte code compilation
26 | Chapter 2: How Python Runs Programs
www.it-ebooks.info
The Python Virtual Machine (PVM)
Performance implications
Program Execution | 27
www.it-ebooks.info
Development implications
eval
exec
28 | Chapter 2: How Python Runs Programs
www.it-ebooks.info
Execution Model Variations
Python Implementation Alternatives
CPython
Jython
Execution Model Variations | 29
www.it-ebooks.info
IronPython
Execution Optimization Tools
The Psyco just-in-time compiler
30 | Chapter 2: How Python Runs Programs
www.it-ebooks.info
The Shedskin C++ translator
Execution Model Variations | 31
www.it-ebooks.info
Frozen Binaries
32 | Chapter 2: How Python Runs Programs
www.it-ebooks.info
Other Execution Options
Future Possibilities?
Execution Model Variations | 33
www.it-ebooks.info
Chapter Summary
Test Your Knowledge: Quiz 1.#code#that#executes#other#code 2.#what#you#actually#write#for#the#program 3.#Is#what#python#compiles 4.#Python#Virtual#Machine,#the#engine# that#interprets#
Test Your Knowledge: Answers the#byte#code.#
5.#IronPython,#JPython#
34 | Chapter 2: How Python Runs Programs
www.it-ebooks.info
CHAPTER 3
How You Run Programs
exec
The Interactive Prompt
python
35
www.it-ebooks.info
% python Python 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)] ... Type "help", "copyright", "credits" or "license" for more information. >>>
python
PATH /usr/local/bin/python C:\Python30\python
/usr/bin/python
C:\misc> c:\python30\python Python 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)] ... Type "help", "copyright", "credits" or "license" for more information. >>>
cd c:\python30 C:\misc> cd C:\Python30 C:\Python30> python Python 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)] ... Type "help", "copyright", "credits" or "license" for more information. >>>
python
36 | Chapter 3: How You Run Programs
www.it-ebooks.info
Running Code Interactively >>> >>> print
print
% python >>> print('Hello world!') Hello world! >>> print(2 ** 8) 256
print >>> 2 ** 8
>>> lumberjack = 'okay' >>> lumberjack 'okay' >>> 2 ** 8 256 >>> %
lumberjack
2 ** 8
print
The Interactive Prompt | 37
www.it-ebooks.info
print
Why the Interactive Prompt?
Experimenting
'Spam!' * 8
>>> 'Spam!' * 8 'Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam!'
*
38 | Chapter 3: How You Run Programs
>>>
www.it-ebooks.info
>>> X Traceback (most recent call last): File "", line 1, in NameError: name 'X' is not defined
Testing
>>> import os >>> os.getcwd() 'c:\\Python30'
Using the Interactive Prompt
os.system
The Interactive Prompt | 39
www.it-ebooks.info
print print print print
...
...
>>>
40 | Chapter 3: How You Run Programs
... sys
>>>
www.it-ebooks.info
Entering multiline statements
for
if
>>> for x in 'spam': ... print(x) ...
>>> for x in 'spam': ... print(x) ... print('done') File "", line 3 print('done') ^ SyntaxError: invalid syntax
System Command Lines and Files
System Command Lines and Files | 41
www.it-ebooks.info
python
A First Script
import sys print(sys.platform) print(2 ** 100) x = 'Spam!' print(x * 8)
print x sys.platform sys
42 | Chapter 3: How You Run Programs
www.it-ebooks.info
# #
Running Files with Command Lines python % python script1.py win32 1267650600228229401496703205376 Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam!
PATH print
% python script1.py > saveit.txt
System Command Lines and Files | 43
www.it-ebooks.info
C:\Python30> python script1.py win32 1267650600228229401496703205376 Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam!
PATH D:\temp> C:\python30\python script1.py win32 1267650600228229401496703205376 Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam!
D:\temp> script1.py
D:\other> python c:\code\otherscript.py
PATH D:\other> C:\Python30\python c:\code\otherscript.py
Using Command Lines and Files
44 | Chapter 3: How You Run Programs
www.it-ebooks.info
python script1.py
python script1
import import script1
python d:\tests\spam.py import spam print print print print
System Command Lines and Files | 45
www.it-ebooks.info
Unix Executable Scripts (#!)
#!
chmod +x file.py
#!/usr/local/bin/python print('The Bright Side ' + 'of Life...')
#
chmod +x brian % brian The Bright Side of Life...
python
brian.py #!
python brian.py #!
46 | Chapter 3: How You Run Programs
www.it-ebooks.info
C:\misc> python brian The Bright Side of Life...
#!
The Unix env Lookup Trick #!/usr/bin/env python ...script goes here...
PATH
PATH
Clicking File Icons
#!
Clicking Icons on Windows
Clicking File Icons | 47
www.it-ebooks.info
import sys print(sys.platform) print(2 ** 100) x = 'Spam!' print(x * 8)
C:\misc> c:\python30\python script1.py win32 1267650600228229401496703205376
48 | Chapter 3: How You Run Programs
www.it-ebooks.info
The input Trick
input raw_input import sys print(sys.platform) print(2 ** 100) x = 'Spam!' print(x * 8) input()
input
Clicking File Icons | 49
www.it-ebooks.info
input print input input('Press Enter to exit') nextinput = input() python spam.py < input.txt
print
input
raw_input()
input() input
input
raw_input input
Other Icon-Click Limitations input
input
50 | Chapter 3: How You Run Programs
eval(input())
www.it-ebooks.info
try
Module Imports and Reloads
import
input C:\misc> c:\python30\python >>> import script1 win32 1267650600228229401496703205376 Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam!
Module Imports and Reloads | 51
www.it-ebooks.info
>>> import script1 >>> import script1
reload imp >>> from imp import reload >>> reload(script1) win32 65536 Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam! >>>
from reload
print 2 ** 16 import
reload
reload reload import reload
import reload reload
52 | Chapter 3: How You Run Programs
www.it-ebooks.info
reload imp imp.reload(M)
from imp import reload import from
import imp reload(M)
reload reload
reload reload
from from import reload
reload
import
module.attribute
The Grander Module Story: Attributes
import
from
reload
title = "The Meaning of Life"
title
Module Imports and Reloads | 53
www.it-ebooks.info
title import % python >>> import myfile >>> print(myfile.title) The Meaning of Life
object.attribute title
myfile
myfile.title from % python >>> from myfile import title >>> print(title) The Meaning of Life
from
import from
title import
myfile.title
from
title
a = 'dead' b = 'parrot' c = 'sketch' print(a, b, c)
print
import
from .py
54 | Chapter 3: How You Run Programs
import
www.it-ebooks.info
% python threenames.py dead parrot sketch
import
from
import from
% python >>> import threenames dead parrot sketch >>> >>> threenames.b, threenames.c ('parrot', 'sketch') >>> >>> from threenames import a, b, c >>> b, c ('parrot', 'sketch')
dir
>>> dir(threenames) ['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'a', 'b', 'c']
dir
a b
c
dir
Modules and namespaces
import
Module Imports and Reloads | 55
www.it-ebooks.info
from from
import
from
from from
import and reload Usage Notes import
reload
reload reload
reload from
reload
56 | Chapter 3: How You Run Programs
www.it-ebooks.info
Using exec to Run Module Files exec(open('module.py').read()) exec C:\misc> c:\python30\python >>> exec(open('script1.py').read()) win32 65536 Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam! ...change script1.py in a text edit window... >>> exec(open('script1.py').read()) win32 4294967296 Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam!
exec exec exec
exec
exec
from x
exec
>>> x = 999 >>> exec(open('script1.py').read()) ...same outout... >>> x 'Spam!'
sys.path PYTHONPATH
sys PYTHONPATH
Using exec to Run Module Files | 57
www.it-ebooks.info
import
execfile('module.py') exec(open('module.py')) exec(open('module.py').read())
exec
exec
The IDLE User Interface exec
IDLE Basics
58 | Chapter 3: How You Run Programs
www.it-ebooks.info
>>>
yum tkinter idle
The IDLE User Interface | 59
www.it-ebooks.info
Using IDLE
60 | Chapter 3: How You Run Programs
www.it-ebooks.info
The IDLE User Interface | 61
www.it-ebooks.info
-n idle.py -n cd
import
Advanced IDLE Tools
62 | Chapter 3: How You Run Programs
www.it-ebooks.info
Other IDEs
Other IDEs | 63
www.it-ebooks.info
Other Launch Options
Embedding Calls
64 | Chapter 3: How You Run Programs
www.it-ebooks.info
#include ... Py_Initialize(); PyRun_SimpleString("x = 'brave ' + 'sir robin'");
Frozen Binary Executables
Text Editor Launch Options
Other Launch Options | 65
www.it-ebooks.info
Still Other Launch Options
os.popen os.system
Future Possibilities?
Which Option Should I Use?
66 | Chapter 3: How You Run Programs
www.it-ebooks.info
Debugging Python Code
print print print
#
print
print
Which Option Should I Use? | 67
www.it-ebooks.info
Chapter Summary exec
Test Your Knowledge: Quiz
unix%open%terminal,%use%IDE3GUI %type%python in%the%directory%you%have%python
icon%clicking,%within%the%text%file%you are%using,%by%using%IDE,%command modules%run%only%once,%even%if% changes%are%made%
open/new%file%then%run3>run%module some%tools%are%not%available a%module%is%a%namespace,%a%package 68 | Chapter 3: How You Run Programs
of%variables.%
www.it-ebooks.info
Test Your Knowledge: Answers python
PATH cd python
C:\Python30\python
exec #!
input
Test Your Knowledge: Answers | 69
www.it-ebooks.info
Test Your Knowledge: Part I Exercises
>>>
"Hello
World!" cd PATH PATH
print('Hello module world!')
python module1.py
exec
>>>
70 | Chapter 3: How You Run Programs
www.it-ebooks.info
#! #!
2 ** 500
1 / 0
L = [1, 2] L.append(L) L
Test Your Knowledge: Part I Exercises | 71
www.it-ebooks.info
help
72 | Chapter 3: How You Run Programs
www.it-ebooks.info
PART II
Types and Operations
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 4
Introducing Python Object Types
75
www.it-ebooks.info
Why Use Built-in Types?
76 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
Python’s Core Data Types
Object type
Example literals/creation
Numbers
1234, 3.1415, 3+4j, Decimal, Fraction
Strings
'spam', "guido's", b'a\x01c'
Lists
[1, [2, 'three'], 4]
Dictionaries
{'food': 'spam', 'taste': 'yum'}
Tuples
(1, 'spam', 4, 'U')
Files
myfile = open('eggs', 'r')
Sets
set('abc'), {'a', 'b', 'c'}
Other core types
Booleans, types, None
Program unit types
Functions, modules, classes (Part IV, Part V, Part VI)
Implementation-related types
Compiled code, stack tracebacks (Part IV, Part VII)
def class import
lambda
>>> 'spam'
const
Why Use Built-in Types? | 77
www.it-ebooks.info
Numbers
+
78 | Chapter 4: Introducing Python Object Types
*
**
www.it-ebooks.info
>>> 123 + 222 345 >>> 1.5 * 4 6.0 >>> 2 ** 100 1267650600228229401496703205376
>>> len(str(2 ** 1000000)) 301030
>>> 3.1415 * 2 6.2830000000000004 >>> print(3.1415 * 2) 6.283
repr
str print
>>> import math >>> math.pi 3.1415926535897931 >>> math.sqrt(85) 9.2195444572928871
math random >>> import random >>> random.random() 0.59268735266273953 >>> random.choice([1, 2, 3, 4]) 1
Numbers | 79
www.it-ebooks.info
Strings
Sequence Operations len >>> >>> 4 >>> 'S' >>> 'p'
S = 'Spam' len(S) S[0] S[1]
S
>>> S[-1] 'm' >>> S[-2] 'a'
80 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
>>> S[-1] 'm' >>> S[len(S)-1] 'm'
>>> S 'Spam' >>> S[1:3] 'pa'
X[I:J] X
I
J
S
>>> S[1:] 'pam' >>> S 'Spam' >>> S[0:3] 'Spa' >>> S[:3] 'Spa' >>> S[:-1] 'Spa' >>> S[:] 'Spam'
>>> S Spam' >>> S + 'xyz'
Strings | 81
www.it-ebooks.info
'Spamxyz' >>> S 'Spam' >>> S * 8 'SpamSpamSpamSpamSpamSpamSpamSpam'
+
+
Immutability
>>> S 'Spam' >>> S[0] = 'z' ...error text omitted... TypeError: 'str' object does not support item assignment >>> S = 'z' + S[1:] >>> S 'zpam'
Type-Specific Methods
82 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
find −1
replace
>>> S.find('pa') 1 >>> S 'Spam' >>> S.replace('pa', 'XYZ') 'SXYZm' >>> S 'Spam'
>>> line = 'aaa,bbb,ccccc,dd' >>> line.split(',') ['aaa', 'bbb', 'ccccc', 'dd'] >>> S = 'spam' >>> S.upper() 'SPAM' >>> S.isalpha() True >>> line = 'aaa,bbb,ccccc,dd\n' >>> line = line.rstrip() >>> line 'aaa,bbb,ccccc,dd'
>>> '%s, eggs, and %s' % ('spam', 'SPAM!') 'spam, eggs, and SPAM!' >>> '{0}, eggs, and {1}'.format('spam', 'SPAM!') 'spam, eggs, and SPAM!'
len(X) X[0]
aString.upper()
Strings | 83
www.it-ebooks.info
Getting Help dir S >>> dir(S) ['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_formatter_field_name_split', '_formatter_parser', 'capitalize', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'index', 'isalnum','isalpha', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
dir help >>> help(S.replace) Help on built-in function replace: replace(...) S.replace (old, new[, count]) -> str Return a copy of S with all occurrences of substring old replaced by new. If the optional argument count is given, only the first count occurrences are replaced.
help
help(S)
84 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
dir
help
Other Ways to Code Strings
>>> S = 'A\nB\tC' >>> len(S) 5 >>> ord('\n') 10 >>> S = 'A\0B\0C' >>> len(S) 5
>>> msg = """ aaaaaaaaaaaaa bbb'''bbbbbbbbbb""bbbbbbb'bbbb cccccccccccccc""" >>> msg '\naaaaaaaaaaaaa\nbbb\'\'\'bbbbbbbbbb""bbbbbbb\'bbbb\ncccccccccccccc'
r str bytes str str bytes
Pattern Matching
Strings | 85
www.it-ebooks.info
re >>> import re >>> match = re.match('Hello[ \t]*(.*)world', 'Hello >>> match.group(1) 'Python '
Python world')
>>> match = re.match('/(.*)/(.*)/(.*)', '/usr/home/lumberjack') >>> match.groups() ('usr', 'home', 'lumberjack')
Lists
Sequence Operations
>>> L = [123, 'spam', 1.23] >>> len(L) 3
>>> L[0] 123 >>> L[:-1] [123, 'spam'] >>> L + [4, 5, 6] [123, 'spam', 1.23, 4, 5, 6]
86 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
>>> L [123, 'spam', 1.23]
Type-Specific Operations
>>> L.append('NI') >>> L [123, 'spam', 1.23, 'NI'] >>> L.pop(2) 1.23 >>> L [123, 'spam', 'NI']
append pop
del
insert
remove
>>> M = ['bb', 'aa', 'cc'] >>> M.sort() >>> M ['aa', 'bb', 'cc'] >>> M.reverse() >>> M ['cc', 'bb', 'aa']
sort reverse
Bounds Checking
>>> L [123, 'spam', 'NI'] >>> L[99] ...error text omitted... IndexError: list index out of range
Lists | 87
www.it-ebooks.info
>>> L[99] = 1 ...error text omitted... IndexError: list assignment index out of range
append
Nesting
>>> M = [[1, 2, [4, 5, [7, 8, >>> M [[1, 2, 3], [4,
3], 6], 9]] 5, 6], [7, 8, 9]]
>>> M[1] [4, 5, 6] >>> M[1][2] 6
Comprehensions
88 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
>>> col2 = [row[1] for row in M] >>> col2 [2, 5, 8] >>> M [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
row row[1]
M
>>> [row[1] + 1 for row in M] [3, 6, 9] >>> [row[1] for row in M if row[1] % 2 == 0] [2, 8]
if
%
>>> diag = [M[i][i] for i in [0, 1, 2]] >>> diag [1, 5, 9] >>> doubles = [c * 2 for c in 'spam'] >>> doubles ['ss', 'pp', 'aa', 'mm']
map
filter
sum
Lists | 89
www.it-ebooks.info
>>> G = (sum(row) for row in M) >>> next(G) 6 >>> next(G) 15
map list >>> list(map(sum, M)) [6, 15, 24]
>>> {sum(row) for row in M} {24, 6, 15} >>> {i : sum(M[i]) for i in range(3)} {0: 6, 1: 15, 2: 24}
>>> [ord(x) for x in 'spaam'] [115, 112, 97, 97, 109] >>> {ord(x) for x in 'spaam'} {112, 97, 115, 109} >>> {x: ord(x) for x in 'spaam'} {'a': 97, 'p': 112, 's': 115, 'm': 109}
Dictionaries
Mapping Operations
>>> D = {'food': 'Spam', 'quantity': 4, 'color': 'pink'}
90 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
>>> D['food'] 'Spam'
'
'
>>> D['quantity'] += 1 ' ' >>> D {'food': 'Spam', 'color': 'pink', 'quantity': 5}
>>> >>> >>> >>>
D = {} D['name'] = 'Bob' D['job'] = 'dev' D['age'] = 40
>>> D {'age': 40, 'job': 'dev', 'name': 'Bob'} >>> print(D['name']) Bob
Nesting Revisited
>>> rec = {'name': {'first': 'Bob', 'last': 'Smith'}, 'job': ['dev', 'mgr'], 'age': 40.5}
Dictionaries | 91
www.it-ebooks.info
>>> rec['name'] {'last': 'Smith', 'first': 'Bob'}
'
'
>>> rec['name']['last'] 'Smith' >>> rec['job'] ['dev', 'mgr'] >>> rec['job'][-1] 'mgr'
'
'
>>> rec['job'].append('janitor') ' >>> rec {'age': 40.5, 'job': ['dev', 'mgr', 'janitor'], 'name': {'last': 'Smith', 'first': 'Bob'}}
>>> rec = 0
rec pickle
92 | Chapter 4: Introducing Python Object Types
shelve
www.it-ebooks.info
Sorting Keys: for Loops
>>> D = {'a': 1, 'b': 2, 'c': 3} >>> D {'a': 1, 'c': 3, 'b': 2}
keys sort
for for
>>> Ks = list(D.keys()) >>> Ks ['a', 'c', 'b'] >>> Ks.sort() >>> Ks ['a', 'b', 'c'] >>> for key in Ks: print(key, '=>', D[key]) a => 1 b => 2 c => 3
sorted sorted >>> D {'a': 1, 'c': 3, 'b': 2} >>> for key in sorted(D): print(key, '=>', D[key]) a => 1 b => 2 c => 3
for for
Dictionaries | 93
www.it-ebooks.info
key
for
while for
>>> for c in 'spam': print(c.upper()) S P A M
while >>> x = 4 >>> while x > 0: print('spam!' * x) x -= 1 spam!spam!spam!spam! spam!spam!spam! spam!spam! spam!
Iteration and Optimization for
iter
next
sorted keys next
94 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
>>> squares = [x ** 2 for x in [1, 2, 3, 4, 5]] >>> squares [1, 4, 9, 16, 25]
for >>> squares = [] >>> for x in [1, 2, 3, 4, 5]: squares.append(x ** 2) >>> squares [1, 4, 9, 16, 25]
map filter
for
time
timeit
profile
Missing Keys: if Tests >>> D {'a': 1, 'c': 3, 'b': 2} >>> D['e'] = 99 >>> D {'a': 1, 'c': 3, 'b': 2, 'e': 99} >>> D['f'] ...error text omitted... KeyError: 'f'
in
Dictionaries | 95
www.it-ebooks.info
if for
if
>>> 'f' in D False >>> if not 'f' in D: print('missing') missing
if if if else
elif
get
has_key try if else if
>>> >>> 0 >>> >>> 0
value = D.get('x', 0) value value = D['x'] if 'x' in D else 0 value
Tuples
>>> T = (1, 2, 3, 4) >>> len(T) 4 >> T + (5, 6) (1, 2, 3, 4, 5, 6) >>> T[0] 1
96 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
>>> T.index(4) 3 >>> T.count(4) 1
>>> T[0] = 2 ...error text omitted... TypeError: 'tuple' object does not support item assignment
>>> T = ('spam', 3.0, [11, 22, 33]) >>> T[1] 3.0 >>> T[2][1] 22 >>> T.append(4) AttributeError: 'tuple' object has no attribute 'append'
Why Tuples?
Files open 'w' >>> >>> 6 >>> 6 >>>
f = open('data.txt', 'w') f.write('Hello\n') f.write('world\n') f.close()
Files | 97
www.it-ebooks.info
'r'
>>> f = open('data.txt') >>> text = f.read() >>> text 'Hello\nworld\n' >>> print(text) Hello world >>> text.split() ['Hello', 'world']
read readline seek for dir
help
>>> dir(f) [ ...many names omitted... 'buffer', 'close', 'closed', 'encoding', 'errors', 'fileno', 'flush', 'isatty', 'line_buffering', 'mode', 'name', 'newlines', 'read', 'readable', 'readline', 'readlines', 'seek', 'seekable', 'tell', 'truncate', 'writable', 'write', 'writelines'] >>>help(f.seek) ...try it and see...
bytes >>> data = open('data.bin', 'rb').read() >>> data b'\x00\x00\x00\x07spam\x00\x08' >>> data[4:8] b'spam'
98 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
Other File-Like Tools open
Other Core Types
set {...} >>> X = set('spam') >>> Y = {'h', 'a', 'm'} >>> X, Y ({'a', 'p', 's', 'm'}, {'a', 'h', 'm'}) >>> X & Y {'a', 'm'} >>> X | Y {'a', 'p', 's', 'h', 'm'} >>> X – Y {'p', 's'} >>> {x ** 2 for x in [1, 2, 3, 4]} {16, 1, 4, 9}
>>> 1 / 3 0.33333333333333331 >>> (2/3) + (1/2)
Other Core Types | 99
www.it-ebooks.info
1.1666666666666665 >>> import decimal >>> d = decimal.Decimal('3.141') >>> d + 1 Decimal('4.141') >>> decimal.getcontext().prec = 2 >>> decimal.Decimal('1.00') / decimal.Decimal('3.00') Decimal('0.33') >>> from fractions import Fraction >>> f = Fraction(2, 3) >>> f + 1 Fraction(5, 3) >>> f + Fraction(1, 2) Fraction(7, 6)
True
False
None >>> 1 > 2, 1 < 2 (False, True) >>> bool('spam') True >>> X = None >>> print(X) None >>> L = [None] * 100 >>> L [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, ...a list of 100 Nones...]
How to Break Your Code’s Flexibility type
L
>>> type(L) >>> type(type(L))
>>> type(L)
100 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
>>> type(type(L))
>>> if type(L) == type([]): print('yes') yes >>> if type(L) == list: print('yes') yes >>> if isinstance(L, list): print('yes') yes
User-Defined Classes
>>> class Worker: def __init__(self, name, pay): self.name = name
Other Core Types | 101
www.it-ebooks.info
self.pay = pay def lastName(self): return self.name.split()[-1] def giveRaise(self, percent): self.pay *= (1.0 + percent)
name
self >>> bob = Worker('Bob Smith', 50000) >>> sue = Worker('Sue Jones', 60000) >>> bob.lastName() 'Smith' >>> sue.lastName() 'Jones' >>> sue.giveRaise(.10) >>> sue.pay 66000.0
name
And Everything Else
102 | Chapter 4: Introducing Python Object Types
Worker pay
pay
www.it-ebooks.info
class
Chapter Summary
Test Your Knowledge: Quiz
Test Your Knowledge: Quiz | 103
www.it-ebooks.info
Test Your Knowledge: Answers None
'spam' open
+
104 | Chapter 4: Introducing Python Object Types
www.it-ebooks.info
CHAPTER 5
Numeric Types
Numeric Type Basics
105
www.it-ebooks.info
Numeric Literals
Literal
Interpretation
1234, −24, 0, 99999999999999
Integers (unlimited size)
1.23, 1., 3.14e-10, 4E210, 4.0e+210
Floating-point numbers
0177, 0x9ff, 0b101010
Octal, hex, and binary literals in 2.6
0o177, 0x9ff, 0b101010
Octal, hex, and binary literals in 3.0
3+4j, 3.0+4.0j, 3J
Complex number literals
e
106 | Chapter 5: Numeric Types
E
www.it-ebooks.info
l
L L
l
L
0x 0 9
0X
A F 0o 0 7
0O
0 0o 0b
0B
0 1 hex(I) oct(I)
bin(I) int(str, base)
imaginarypart imaginarypart
j
J
realpart+imaginarypart realpart
complex(real, imag)
Numeric Type Basics | 107
www.it-ebooks.info
Built-in Numeric Tools
+ - * / >> ** & pow abs round int hex bin random math
as_integer_ratio is_integer bit_length
Python Expression Operators
X X + Y
+ X
Y X
Y
Y + − * / %
= y
Magnitude comparison, set subset and superset;
x == y, x != y
Value equality operators
x | y
Bitwise OR, set union
x ^ y
Bitwise XOR, set symmetric difference
x & y
Bitwise AND, set intersection
x > y
Shift x left or right by y bits
x + y
Addition, concatenation;
x – y
Subtraction, set difference
x * y
Multiplication, repetition;
x % y
Remainder, format;
x / y, x // y
Division: true and floor
−x, +x
Negation, identity
˜x
Bitwise NOT (inversion)
x ** y
Power (exponentiation)
x[i]
Indexing (sequence, mapping, others)
x[i:j:k]
Slicing
x(...)
Call (function, method, class, other callable)
x.attr
Attribute reference
(...)
Tuple, expression, generator expression
[...]
List, list comprehension
{...}
Dictionary, set, set and dictionary comprehensions
Numeric Type Basics | 109
www.it-ebooks.info
X != Y
X Y
X != Y `X` str
repr(X)
repr
X // Y X / Y
[...]
(...)
{...}
yield if
if else send(...) yield X < Y < Z
X < Y and Y < X X[I:J:K] X[slice(I, J, K)]
sorted(dict.items())
110 | Chapter 5: Numeric Types
www.it-ebooks.info
Mixed operators follow operator precedence
A * B + C * D
X + Y * Z X
(Y * Z) +
* A * B
C * D
Parentheses group subexpressions
X + Y * Z (X + Y) * Z X + (Y * Z)
+
X
Y *
Mixed types are converted up
40 + 3.14
Numeric Type Basics | 111
www.it-ebooks.info
>>> int(3.1415) 3 >>> float(3) 3.0
Preview: Operator overloading and polymorphism
+
112 | Chapter 5: Numeric Types
[i]
www.it-ebooks.info
+ +
Numbers in Action
Variables and Basic Expressions a
b
a
b
% python >>> a = 3 >>> b = 4
#
#
Numbers in Action | 113
www.it-ebooks.info
a
b
3
>>> a + 1, a (4, 2) >>> b * 3, b (12, 2.0) >>> a % 2, b (1, 16) >>> 2 + 4.0, (6.0, 16.0)
4
– 1 / 2 ** 2 2.0 ** b
a
b
>>> c * 2 Traceback (most recent call last): File "", line 1, in ? NameError: name 'c' is not defined
>>> b / 2 + a 5.0 >>> print(b / (2.0 + a)) 0.8
/ +
b // 2 + a + / 2.0
114 | Chapter 5: Numeric Types
www.it-ebooks.info
a
3.0
+ 4 / 5
0
0.8
Numeric Display Formats print print >>> b / (2.0 + a) 0.80000000000000004 >>> print(b / (2.0 + a)) 0.8
print
print
>>> 1 / 2.0 0.5
print >>> num = 1 / 3.0 >>> num 0.33333333333333331 >>> print(num) 0.333333333333 >>> '%e' % num '3.333333e-001' >>> '%4.2f' % num '0.33' >>> '{0:4.2f}'.format(num) '0.33'
Numbers in Action | 115
www.it-ebooks.info
str and repr Display Formats print repr
str
>>> num = 1 / 3 >>> repr(num) '0.33333333333333331' >>> str(num) '0.333333333333'
repr str print str
repr
str
Comparisons: Normal and Chained
>>> 1 < True >>> 2.0 True >>> 2.0 True >>> 2.0 False
2 >= 1 == 2.0 != 2.0
B C)
116 | Chapter 5: Numeric Types
A
C
(A < B < C) (A < B and B <
www.it-ebooks.info
>>> X = 2 >>> Y = 4 >>> Z = 6
Y >>> X < Y < Z True >>> X < Y and Y < Z True
>>> X < Y > Z False >>> X < Y and Y > Z False >>> 1 < 2 < 3.0 < 4 True >>> 1 > 2 > 3.0 > 4 False
>>> 1 == 2 < 3 False
0 < 3
1 == 2 False True
True
False
Division: Classic, Floor, and True
X / Y
X // Y
Numbers in Action | 117
www.it-ebooks.info
/
//
/ //
/ //
C:\misc> C:\Python30\python >>> >>> 10 / 4 2.5 >>> 10 // 4 2 >>> 10 / 4.0 2.5 >>> 10 // 4.0 2.0 C:\misc> C:\Python26\python >>> >>> 10 / 4 2 >>> 10 // 4 2 >>> 10 / 4.0 2.5 >>> 10 // 4.0 2.0
// / //
118 | Chapter 5: Numeric Types
www.it-ebooks.info
Supporting either Python / // float
/
X = Y // Z X = Y / float(Z)
/
__future__
float C:\misc> C:\Python26\python >>> from __future__ import division >>> 10 / 4 2.5 >>> 10 // 4 2
Floor versus truncation //
math >>> >>> 2 >>> -3 >>> 2 >>> -2
import math math.floor(2.5) math.floor(-2.5) math.trunc(2.5) math.trunc(-2.5)
C:\misc> c:\python30\python >>> 5 / 2, 5 / −2 (2.5, −2.5) >>> 5 // 2, 5 // −2 (2, −3) >>> 5 / 2.0, 5 / −2.0 (2.5, −2.5)
Numbers in Action | 119
www.it-ebooks.info
>>> 5 // 2.0, 5 // −2.0 (2.0, −3.0)
/ C:\misc> c:\python26\python >>> 5 / 2, 5 / −2 (2, −3) >>> 5 // 2, 5 // −2 (2, −3) >>> 5 / 2.0, 5 / −2.0 (2.5, −2.5) >>> 5 // 2.0, 5 // −2.0 (2.0, −3.0)
math.trunc C:\misc> c:\python30\python >>> import math >>> 5 / −2 −2.5 >>> 5 // −2 -3 >>> math.trunc(5 / −2) −2 C:\misc> c:\python26\python >>> import math >>> 5 / float(−2) −2.5 >>> 5 / −2, 5 // −2 (−3, −3) >>> math.trunc(5 / float(−2)) −2
Why does truncation matter? >>> (5 / 2), (5 / 2.0), (5 / −2.0), (5 / −2) (2.5, 2.5, −2.5, −2.5) >>> (5 // 2), (5 // 2.0), (5 // −2.0), (5 // −2) (2, 2.0, −3.0, −3) >>> (9 / 3), (9.0 / 3), (9 // 3), (9 // 3.0) (3.0, 3.0, 3, 3.0)
>>> (5 / 2), (5 / 2.0), (5 / −2.0), (5 / −2) (2, 2.5, −2.5, −3)
120 | Chapter 5: Numeric Types
round
www.it-ebooks.info
>>> (5 // 2), (5 // 2.0), (5 // −2.0), (5 // −2) (2, 2.0, −3.0, −3) >>> (9 / 3), (9.0 / 3), (9 // 3), (9 // 3.0) (3, 3.0, 3, 3.0)
/ //
while /
from
Integer Precision
>>> 999999999999999999999999999999 + 1 1000000000000000000000000000000
>>> 999999999999999999999999999999 + 1 1000000000000000000000000000000L
>>> 2 ** 200 1606938044258990275541962092341162602522202993782792835301376 >>> 2 ** 200 1606938044258990275541962092341162602522202993782792835301376L
Numbers in Action | 121
www.it-ebooks.info
Complex Numbers
j
J + 2
−3
2 + −3j >>> 1j * 1J (-1+0j) >>> 2 + 1j * 3 (2+3j) >>> (2 + 1j) * 3 (6+3j)
cmath math
Hexadecimal, Octal, and Binary Notation
>>> (1, >>> (1, >>> (1,
0o1, 0o20, 0o377 16, 255) 0x01, 0x10, 0xFF 16, 255) 0b1, 0b10000, 0b11111111 16, 255)
0o377 255 >>> oct(64), hex(64), bin(64) ('0100', '0x40', '0b1000000')
122 | Chapter 5: Numeric Types
0xFF
0b11111111
www.it-ebooks.info
oct
hex
bin
int >>> int('64'), int('100', 8), int('40', 16), int('1000000', 2) (64, 64, 64, 64) >>> int('0x40', 16), int('0b1000000', 2) (64, 64)
eval
>>> eval('64'), eval('0o100'), eval('0x40'), eval('0b1000000') (64, 64, 64, 64)
>>> '{0:o}, {1:x}, {2:b}'.format(64, 64, 64) '100, 40, 1000000' >>> '%o, %x, %X' % (64, 255, 255) '100, ff, FF'
>>> (1, >>> (1,
0o1, 0o20, 0o377 16, 255) 01, 020, 0377 16, 255)
010 0o010
>>> X = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFF >>> X 5192296858534827628530496329220095L >>> oct(X)
Numbers in Action | 123
www.it-ebooks.info
'017777777777777777777777777777777777777L' >>> bin(X) '0b1111111111111111111111111111111111111111111111111111111111 ...and so on...
Bitwise Operations
>>> >>> 4 >>> 3 >>> 1
x = 1 x >> X = 0b0001 >>> X >> bin(X >> bin(X | 0b010) '0b11' >>> bin(X & 0b1) '0b1' >>> X = 0xFF >>> bin(X) '0b11111111' >>> X ^ 0b10101010 85 >>> bin(X ^ 0b10101010) '0b1010101' >>> int('1010101', 2) 85 >>> hex(85) '0x55'
124 | Chapter 5: Numeric Types
0001 0011
www.it-ebooks.info
bit_length
bin
len
>>> X = 99 >>> bin(X), X.bit_length() ('0b1100011', 7) >>> bin(256), (256).bit_length() ('0b100000000', 9) >>> len(bin(256)) - 2 9
Other Built-in Numeric Tools pow
abs
math >>> import math >>> math.pi, math.e (3.1415926535897931, 2.7182818284590451) >>> math.sin(2 * math.pi / 180) 0.034899496702500969 >>> math.sqrt(144), math.sqrt(2) (12.0, 1.4142135623730951) >>> pow(2, 4), 2 ** 4 (16, 16) >>> abs(-42.0), sum((1, 2, 3, 4)) (42.0, 10) >>> min(3, 1, 2, 4), max(3, 1, 2, 4) (1, 4)
sum
min
max
Numbers in Action | 125
www.it-ebooks.info
>>> math.floor(2.567), math.floor(-2.567) (2, −3) >>> math.trunc(2.567), math.trunc(−2.567) (2, −2) >>> int(2.567), int(−2.567) (2, −2) >>> round(2.567), round(2.467), round(2.567, 2) (3, 2, 2.5699999999999998) >>> '%.1f' % 2.567, '{0:.2f}'.format(2.567) ('2.6', '2.57')
(3, 2, 2.57)
print round
>>> (1 / 3), round(1 / 3, 2), ('%.2f' % (1 / 3)) (0.33333333333333331, 0.33000000000000002, '0.33')
>>> import math >>> math.sqrt(144) 12.0 >>> 144 ** .5 12.0 >>> pow(144, .5) 12.0 >>> math.sqrt(1234567890) 35136.418286444619 >>> 1234567890 ** .5 35136.418286444619 >>> pow(1234567890, .5) 35136.418286444619
math abs
round
builtins
126 | Chapter 5: Numeric Types
__builtin__
www.it-ebooks.info
random
>>> import random >>> random.random() 0.44694718823781876 >>> random.random() 0.28970426439292829 >>> random.randint(1, 10) 5 >>> random.randint(1, 10) 4 >>> random.choice(['Life of Brian', 'Holy Grail', 'Meaning of Life']) 'Life of Brian' >>> random.choice(['Life of Brian', 'Holy Grail', 'Meaning of Life']) 'Holy Grail'
random
Other Numeric Types
Decimal Type Decimal
Other Numeric Types | 127
www.it-ebooks.info
The basics
>>> 0.1 + 0.1 + 0.1 - 0.3 5.5511151231257827e-17
>>> print(0.1 + 0.1 + 0.1 - 0.3) 5.55111512313e-17
>>> from decimal import Decimal >>> Decimal('0.1') + Decimal('0.1') + Decimal('0.1') - Decimal('0.3') Decimal('0.0')
Decimal decimal str
>>> Decimal('0.1') + Decimal('0.10') + Decimal('0.10') - Decimal('0.30') Decimal('0.00')
decimal.Decimal.from_float(1.25)
Setting precision globally decimal
>>> import decimal >>> decimal.Decimal(1) / decimal.Decimal(7) Decimal('0.1428571428571428571428571429') >>> decimal.getcontext().prec = 4 >>> decimal.Decimal(1) / decimal.Decimal(7) Decimal('0.1429')
128 | Chapter 5: Numeric Types
www.it-ebooks.info
>>> 1999 + 1.33 2000.3299999999999 >>> >>> decimal.getcontext().prec = 2 >>> pay = decimal.Decimal(str(1999 + 1.33)) >>> pay Decimal('2000.33')
Decimal context manager with C:\misc> C:\Python30\python >>> import decimal >>> decimal.Decimal('1.00') / decimal.Decimal('3.00') Decimal('0.3333333333333333333333333333') >>> >>> with decimal.localcontext() as ctx: ... ctx.prec = 2 ... decimal.Decimal('1.00') / decimal.Decimal('3.00') ... Decimal('0.33') >>> >>> decimal.Decimal('1.00') / decimal.Decimal('3.00') Decimal('0.3333333333333333333333333333')
with
Fraction Type Fraction
The basics Fraction
Decimal
Other Numeric Types | 129
www.it-ebooks.info
Decimal Fraction >>> from fractions import Fraction >>> x = Fraction(1, 3) >>> y = Fraction(4, 6) >>> x Fraction(1, 3) >>> y Fraction(2, 3) >>> print(y) 2/3
Fraction >>> x + y Fraction(1, 1) >>> x – y Fraction(-1, 3) >>> x * y Fraction(2, 9)
Fraction >>> Fraction('.25') Fraction(1, 4) >>> Fraction('1.25') Fraction(5, 4) >>> >>> Fraction('.25') + Fraction('1.25') Fraction(3, 2)
Numeric accuracy
>>> a = 1 / 3.0 >>> b = 4 / 6.0 >>> a 0.33333333333333331 >>> b 0.66666666666666663 >>> a + b 1.0 >>> a - b -0.33333333333333331 >>> a * b 0.22222222222222221
Fraction 130 | Chapter 5: Numeric Types
www.it-ebooks.info
Decimal
>>> 0.1 + 0.1 + 0.1 - 0.3 5.5511151231257827e-17 >>> from fractions import Fraction >>> Fraction(1, 10) + Fraction(1, 10) + Fraction(1, 10) - Fraction(3, 10) Fraction(0, 1) >>> from decimal import Decimal >>> Decimal('0.1') + Decimal('0.1') + Decimal('0.1') - Decimal('0.3') Decimal('0.0')
>>> 1 / 3 0.33333333333333331 >>> Fraction(1, 3) Fraction(1, 3) >>> import decimal >>> decimal.getcontext().prec = 2 >>> decimal.Decimal(1) / decimal.Decimal(3) Decimal('0.33')
>>> (1 / 3) + (6 / 12) 0.83333333333333326 >>> Fraction(6, 12) Fraction(1, 2) >>> Fraction(1, 3) + Fraction(6, 12) Fraction(5, 6) >>> decimal.Decimal(str(1/3)) + decimal.Decimal(str(6/12)) Decimal('0.83') >>> 1000.0 / 1234567890 8.1000000737100011e-07 >>> Fraction(1000, 1234567890) Fraction(100, 123456789)
Conversions and mixed types from_float
Other Numeric Types | 131
www.it-ebooks.info
float
Fraction *
>>> (2.5).as_integer_ratio() (5, 2) >>> f = 2.5 >>> z = Fraction(*f.as_integer_ratio()) >>> z Fraction(5, 2) >>> x Fraction(1, 3) >>> x + z Fraction(17, 6) >>> float(x) 0.33333333333333331 >>> float(z) 2.5 >>> float(x + z) 2.8333333333333335 >>> 17 / 6 2.8333333333333335 >>> Fraction.from_float(1.75) Fraction(7, 4) >>> Fraction(*(1.75).as_integer_ratio()) Fraction(7, 4)
Fraction
>>> x Fraction(1, 3) >>> x + 2 Fraction(7, 3) >>> x + 2.0 2.3333333333333335 >>> x + (1./3) 0.66666666666666663 >>> x + (4./3) 1.6666666666666665 >>> x + Fraction(4, 3) Fraction(5, 3)
132 | Chapter 5: Numeric Types
www.it-ebooks.info
>>> 4.0 / 3 1.3333333333333333 >>> (4.0 / 3).as_integer_ratio() (6004799503160661, 4503599627370496) >>> x Fraction(1, 3) >>> a = x + Fraction(*(4.0 / 3).as_integer_ratio()) >>> a Fraction(22517998136852479, 13510798882111488) >>> 22517998136852479 / 13510798882111488. 1.6666666666666667 >>> a.limit_denominator(10) Fraction(5, 3)
Fraction
Sets
Set basics in Python 2.6
set >>> x = set('abcde') >>> y = set('bdxyz')
Other Numeric Types | 133
www.it-ebooks.info
>>> x set(['a', 'c', 'b', 'e', 'd'])
>>> 'e' in x True >>> x – y set(['a', 'c', 'e']) >>> x | y set(['a', 'c', 'b', 'e', 'd', 'y', 'x', 'z']) >>> x & y set(['b', 'd']) >>> x ^ y set(['a', 'c', 'e', 'y', 'x', 'z']) >>> x > y, x < y (False, False)
add update
remove
dir
set
x
y >>> z = x.intersection(y) >>> z set(['b', 'd']) >>> z.add('SPAM') >>> z set(['b', 'd', 'SPAM']) >>> z.update(set(['X', 'Y'])) >>> z set(['Y', 'X', 'b', 'd', 'SPAM']) >>> z.remove('b') >>> z set(['Y', 'X', 'd', 'SPAM'])
len for
>>> for item in set('abc'): print(item * 3) ... aaa
134 | Chapter 5: Numeric Types
www.it-ebooks.info
ccc bbb
>>> S = set([1, 2, 3]) >>> S | set([3, 4]) set([1, 2, 3, 4]) >>> S | [3, 4] TypeError: unsupported operand type(s) for |: 'set' and 'list' >>> S.union([3, 4]) set([1, 2, 3, 4]) >>> S.intersection((1, 3, 5)) set([1, 3]) >>> S.issubset(range(-5, 5)) True
Set literals in Python 3.0 set
set([1, 2, 3, 4]) {1, 2, 3, 4}
set
C:\Misc> c:\python30\python >>> set([1, 2, 3, 4]) {1, 2, 3, 4} >>> set('spam') {'a', 'p', 's', 'm'} >>> {1, 2, 3, 4}
Other Numeric Types | 135
www.it-ebooks.info
{1, 2, 3, 4} >>> S = {'s', 'p', 'a', 'm'} >>> S.add('alot') >>> S {'a', 'p', 's', 'm', 'alot'}
>>> S1 = {1, 2, 3, 4} >>> S1 & {1, 3} {1, 3} >>> {1, 5, 3, 6} | S1 {1, 2, 3, 4, 5, 6} >>> S1 - {1, 3, 4} {2} >>> S1 > {1, 3} True
{} >>> S1 - {1, 2, 3, 4} set() >>> type({}) >>> S = set() >>> S.add(1.23) >>> S {1.23}
>>> {1, 2, 3} | {3, 4} {1, 2, 3, 4} >>> {1, 2, 3} | [3, 4] TypeError: unsupported operand type(s) for |: 'set' and 'list' >>> {1, >>> {1, >>> {1,
{1, 2, 3}.union([3, 4]) 2, 3, 4} {1, 2, 3}.union({3, 4}) 2, 3, 4} {1, 2, 3}.union(set([3, 4])) 2, 3, 4}
>>> {1, 2, 3}.intersection((1, 3, 5)) {1, 3} >>> {1, 2, 3}.issubset(range(-5, 5)) True
Immutable constraints and frozen sets
136 | Chapter 5: Numeric Types
set
www.it-ebooks.info
>>> S {1.23} >>> S.add([1, 2, 3]) TypeError: unhashable type: 'list' >>> S.add({'a':1}) TypeError: unhashable type: 'dict' >>> S.add((1, 2, 3)) >>> S {1.23, (1, 2, 3)} >>> S | {(4, 5, 6), (1, 2, 3)} {1.23, (4, 5, 6), (1, 2, 3)} >>> (1, 2, 3) in S True >>> (1, 4, 3) in S False
frozenset
set
Set comprehensions in Python 3.0
>>> {x ** 2 for x in [1, 2, 3, 4]} {16, 1, 4, 9}
x ** 2
>>> {x for x in 'spam'} {'a', 'p', 's', 'm'} >>> {c * 4 for c in 'spam'} {'ssss', 'aaaa', 'pppp', 'mmmm'} >>> {c * 4 for c in 'spamham'}
Other Numeric Types | 137
www.it-ebooks.info
{'ssss', 'aaaa', 'hhhh', 'pppp', 'mmmm'} >>> S = {c * 4 for c in 'spam'} >>> S | {'mmmm', 'xxxx'} {'ssss', 'aaaa', 'pppp', 'mmmm', 'xxxx'} >>> S & {'mmmm', 'xxxx'} {'mmmm'}
if
Why sets?
list >>> >>> {1, >>> >>> [1,
L = [1, 2, 1, 3, 2, 4, 5] set(L) 2, 3, 4, 5} L = list(set(L)) L 2, 3, 4, 5]
set >>> engineers = {'bob', 'sue', 'ann', 'vic'} >>> managers = {'tom', 'sue'} >>> 'bob' in engineers True >>> engineers & managers
138 | Chapter 5: Numeric Types
www.it-ebooks.info
{'sue'} >>> engineers | managers {'vic', 'sue', 'tom', 'bob', 'ann'} >>> engineers – managers {'vic', 'bob', 'ann'} >>> managers – engineers {'tom'} >>> engineers > managers False >>> {'bob', 'sue'} < engineers True >>> (managers | engineers) > managers True >>> managers ^ engineers {'vic', 'bob', 'ann', 'tom'} >>> (managers | engineers) - (managers ^ engineers) {'sue'}
Booleans bool True
False
bool True
True False False
bool int True
str
True False repr True
False 1
0 bool
False
1
0 while True:
while 1:
Other Numeric Types | 139
www.it-ebooks.info
flag = False True True
False
1
0
1 bool
True True + 4
5
>>> type(True) >>> isinstance(True, int) True >>> True == 1 True >>> True is 1 False >>> True or False True >>> True + 4 5
and
Numeric Extensions
140 | Chapter 5: Numeric Types
False
0
or
1
www.it-ebooks.info
Chapter Summary
Test Your Knowledge: Quiz 2 * (3 + 4) 2 * 3 + 4 2 + 3 * 4 1 + 2.0 + 3
Test Your Knowledge: Answers 14 10
14
math
math math.sqrt(N)
Test Your Knowledge: Answers | 141
www.it-ebooks.info
X ** 2
pow(X, 2) 0.5
int(N)
math.trunc(N)
X ** .5
round(N, digits) math.floor(N)
float(I) / oct(I)
hex(I) bin(I) %
format
int(S, base) 8 16
142 | Chapter 5: Numeric Types
2
eval(S)
www.it-ebooks.info
CHAPTER 6
The Dynamic Typing Interlude
The Case of the Missing Declaration Statements a = 3 a a
143
www.it-ebooks.info
Variables, Objects, and References a = 3 a
a
a
>>> a = 3
3 a a
144 | Chapter 6: The Dynamic Typing Interlude
3
www.it-ebooks.info
0
Types Live with Objects, Not Variables
The Case of the Missing Declaration Statements | 145
www.it-ebooks.info
>>> a = 3 >>> a = 'spam' >>> a = 1.23
a a a = 'spam'
a a
3 3 int 'spam'
str
Objects Are Garbage-Collected a
3 >>> a = 3 >>> a = 'spam'
x
146 | Chapter 6: The Dynamic Typing Interlude
www.it-ebooks.info
>>> >>> >>> >>>
x x x x
= = = =
42 'shrubbery' 3.1415 [1, 2, 3]
x x
x 'shrubbery'
42
x
gc
The Case of the Missing Declaration Statements | 147
www.it-ebooks.info
Shared References
>>> a = 3 >>> b = a
b
a 3 a
b
b
>>> a = 3 >>> b = a >>> a = 'spam'
'spam' b b
a 3 b
b
a
>>> a = 3 >>> b = a >>> a = a + 2
148 | Chapter 6: The Dynamic Typing Interlude
'spam'
www.it-ebooks.info
a 3
a
b a
5
+
b 3
Shared References and In-Place Changes
>>> L1 = [2, 3, 4] >>> L2 = L1
Shared References | 149
www.it-ebooks.info
L1
2 3 L1[0] L1
4
2
L1
L2
a
b
>>> L1 = 24
L1
L2
>>> L1 = [2, 3, 4] >>> L2 = L1 >>> L1[0] = 24 >>> L1 [24, 3, 4] >>> L2 [24, 3, 4]
L1 L1 L1 L2
L1 L2
list copy >>> L1 = [2, 3, 4] >>> L2 = L1[:] >>> L1[0] = 24 >>> L1 [24, 3, 4] >>> L2 [2, 3, 4]
L1 L1
150 | Chapter 6: The Dynamic Typing Interlude
L2
L2
www.it-ebooks.info
X.copy()
copy
import copy X = copy.copy(Y) X = copy.deepcopy(Y)
class
Shared References and Equality
>>> x = 42 >>> x = 'shrubbery'
42 42
>>> L >>> M >>> L True >>> L True
= [1, 2, 3] = L == M is M
== is
True
Shared References | 151
www.it-ebooks.info
is False
>>> L >>> M >>> L True >>> L False
= [1, 2, 3] = [1, 2, 3] == M
>>> X >>> Y >>> X True >>> X True
= 42 = 42 == Y
is M
is Y
X
Y
==
is
is getrefcount
sys 1
>>> import sys >>> sys.getrefcount(1) 837
is
Dynamic Typing Is Everywhere
152 | Chapter 6: The Dynamic Typing Interlude
www.it-ebooks.info
for
Chapter Summary
Test Your Knowledge: Quiz A A = "spam" B = A B = "shrubbery"
A A = ["spam"] B = A B[0] = "shrubbery"
A A = ["spam"] B = A[:] B[0] = "shrubbery"
Test Your Knowledge: Quiz | 153
www.it-ebooks.info
Test Your Knowledge: Answers A
"spam"
B
"shrubbery"
B
A
B
"spam" B A
B = B +
'shrubbery' B
A
A B
["shrubbery"] B B
A
A
A ["spam"]
B
B ==
is B
A
154 | Chapter 6: The Dynamic Typing Interlude
www.it-ebooks.info
CHAPTER 7
Strings
155
www.it-ebooks.info
Operation
Interpretation
S = ''
Empty string
S = "spam's"
Double quotes, same as single
S = 's\np\ta\x00m'
Escape sequences
S = """..."""
Triple-quoted block strings
S = r'\temp\spam'
Raw strings
S = b'spam'
Byte strings in 3.0 (Chapter 36)
S = u'spam'
Unicode strings in 2.6 only (Chapter 36)
S1 + S2
Concatenate, repeat
S * 3 S[i]
Index, slice, length
S[i:j] len(S) "a %s parrot" % kind
String formatting expression
"a {0} parrot".format(kind)
String formatting method in 2.6 and 3.0
S.find('pa')
String method calls: search,
S.rstrip()
remove whitespace,
S.replace('pa', 'xx')
replacement,
S.split(',')
split on delimiter,
S.isdigit()
content test,
S.lower()
case conversion,
S.endswith('spam')
end test,
'spam'.join(strlist)
delimiter join,
S.encode('latin-1')
Unicode encoding, etc.
for x in S: print(x)
Iteration, membership
'spam' in S [c * 2 for c in S] map(ord, S)
re
156 | Chapter 7: Strings
www.it-ebooks.info
str
str bytes bytearray
bytes
unicode str bytearray
String Literals 'spa"m' "spa'm" '''... spam ...''' """... spam ...""" "s\tp\na\0m" r"C:\new\test.spm"
String Literals | 157
www.it-ebooks.info
b'sp\x01am' u'eggs\u0020spam'
Single- and Double-Quoted Strings Are the Same
>>> 'shrubbery', "shrubbery" ('shrubbery', 'shrubbery')
>>> 'knight"s', "knight's" ('knight"s', "knight's")
+
>>> title = "Meaning " 'of' " Life" >>> title 'Meaning of Life'
>>> 'knight\'s', "knight\"s" ("knight's", 'knight"s')
Escape Sequences Represent Special Bytes
\
158 | Chapter 7: Strings
www.it-ebooks.info
>>> s = 'a\nb\tc'
\n \t print >>> s 'a\nb\tc' >>> print(s) a b c
len >>> len(s) 5
Escape
Meaning
\newline
Ignored (continuation line)
\\
Backslash (stores one \)
\'
Single quote (stores ')
\"
Double quote (stores ")
\a
Bell
\b
Backspace
\f
Formfeed
\n
Newline (linefeed)
\r
Carriage return
\t
Horizontal tab
\v
Vertical tab
\xhh
Character with hex value hh (at most 2 digits)
\ooo
Character with octal value ooo (up to 3 digits)
\0
Null: binary 0 character (doesn’t end string)
String Literals | 159
www.it-ebooks.info
a
Escape
Meaning
\N{ id }
Unicode database ID
\uhhhh
Unicode 16-bit hex
\Uhhhhhhhh
Unicode 32-bit hexa
\other
Not an escape (keeps both \ and other)
The \Uhhhh... escape sequence takes exactly eight hexadecimal digits (h); both \u and \U can be used only in Unicode string literals.
>>> s = 'a\0b\0c' >>> s 'a\x00b\x00c' >>> len(s) 5
>>> s = '\001\002\x03' >>> s '\x01\x02\x03' >>> len(s) 3
>>> S = "s\tp\na\x00m" >>> S 's\tp\na\x00m' >>> len(S) 7 >>> print(S) s p a m
b
'rb' 'wb'
bytes str
struct
160 | Chapter 7: Strings
www.it-ebooks.info
\ >>> x = "C:\py\code" >>> x 'C:\\py\\code' >>> len(x) 10
\\
\
Raw Strings Suppress Escapes
myfile = open('C:\new\text.dat', 'w')
\n
\t
r
r myfile = open(r'C:\new\text.dat', 'w')
myfile = open('C:\\new\\text.dat', 'w')
>>> path = r'C:\new\text.dat' >>> path 'C:\\new\\text.dat' >>> print(path)
String Literals | 161
www.it-ebooks.info
C:\new\text.dat >>> len(path) 15
print len print(path)
re 'C:/new/ text.dat'
r"...\"
r'1\nb\tc\\'[:-1] '1\\nb\\tc\\'
Triple Quotes Code Multiline Block Strings
>>> mantra = """Always look ... on the bright ... side of life.""" >>> >>> mantra 'Always look\n on the bright\nside of life.'
162 | Chapter 7: Strings
r'1\nb\tc' + '\\'
www.it-ebooks.info
... \n
>>> print(mantra) Always look on the bright side of life.
X = 1 """ import os print(os.getcwd()) """ Y = 2
Strings in Action
Strings in Action | 163
www.it-ebooks.info
Basic Operations + * % python >>> len('abc') 3 >>> 'abc' + 'def' 'abcdef' >>> 'Ni!' * 4 'Ni!Ni!Ni!Ni!'
len
>>> print('------- ...more... ---') >>> print('-' * 80)
+ *
+
'abc'+9
9 for
in in
str.find() >>> myjob = "hacker" >>> for c in myjob: print(c, end=' ') ...
164 | Chapter 7: Strings
www.it-ebooks.info
h a c k e r >>> "k" in myjob True >>> "z" in myjob False >>> 'spam' in 'abcspamdef' True
for c
Indexing and Slicing
>>> S = 'spam' >>> S[0], S[−2] ('s', 'a') >>> S[1:3], S[1:], S[:−1] ('pa', 'pam', 'spa')
S S[0] 's'
S[−2]
Strings in Action | 165
www.it-ebooks.info
S[1:3] S[1:] S[:−1]
S[i]
S[0] S[−2] S[i:j]
S[1:3] S[1:]
166 | Chapter 7: Strings
S[len(S)−2]
www.it-ebooks.info
S[:3] S[:−1] S[:] S
Extended slicing: the third limit and slice objects
X[I:J:K] X
I
X[1:10:2]
J
K
X X[::2]
>>> S = 'abcdefghijklmnop' >>> S[1:10:2] 'bdfhj' >>> S[::2] 'acegikmo'
"hello"[::−1] "olleh"
>>> S = 'hello' >>> S[::−1] 'olleh'
S[5:1:−1]
Strings in Action | 167
www.it-ebooks.info
>>> S = 'abcedfg' >>> S[5:1:−1] 'fdec'
for
>>> 'spam'[1:3] 'pa' >>> 'spam'[slice(1, 3)] 'pa' >>> 'spam'[::-1] 'maps' >>> 'spam'[slice(None, None, −1)] 'maps'
Why You Will Care: Slices
argv sys import sys print(sys.argv) % python echo.py −a −b −c ['echo.py', '−a', '−b', '−c']
sys.argv[1:] ['−a', '−b', '−c']
\n line[:−1]
168 | Chapter 7: Strings
www.it-ebooks.info
line.rstrip
String Conversion Tools
>>> "42" + 1 TypeError: cannot concatenate 'str' and 'int' objects
+
>>> int("42"), str(42) (42, '42') >>> repr(42) '42'
int
str repr
print >>> print(str('spam'), repr('spam')) ('spam', "'spam'")
int
str +
>>> S = "42" >>> I = 1 >>> S + I TypeError: cannot concatenate 'str' and 'int' objects >>> int(S) + I 43
Strings in Action | 169
www.it-ebooks.info
>>> S + str(I) '421'
>>> str(3.1415), float("1.5") ('3.1415', 1.5) >>> text = "1.234E-10" >>> float(text) 1.2340000000000001e-010
eval int float
Character code conversions ord chr
>>> ord('s') 115 >>> chr(115) 's'
>>> >>> >>> '6' >>> >>> '7'
S = '5' S = chr(ord(S) + 1) S S = chr(ord(S) + 1) S
int >>> int('5') 5 >>> ord('5') - ord('0') 5
170 | Chapter 7: Strings
www.it-ebooks.info
>>> >>> >>> ... ... ... >>> 13
B = '1101' I = 0 while B != '': I = I * 2 + (ord(B[0]) - ord('0')) B = B[1:] I
I >> int('1101', 2) 13 >>> bin(13) '0b1101'
Changing Strings >>> S = 'spam' >>> S[0] = "x" Raises an error!
>>> S = S + 'SPAM!' >>> S 'spamSPAM!' >>> S = S[:4] + 'Burger' + S[−1] >>> S 'spamBurger!'
S S
replace
Strings in Action | 171
www.it-ebooks.info
>>> S = 'splot' >>> S = S.replace('pl', 'pamal') >>> S 'spamalot'
>>> 'That is %d %s bird!' % (1, 'dead') That is 1 dead bird! >>> 'That is {0} {1} bird!'.format(1, 'dead') 'That is 1 dead bird!'
bytearray bytearray
ord
String Methods
count
172 | Chapter 7: Strings
chr
www.it-ebooks.info
object.attribute
attribute
object function(arguments) argument
function function
object.method(arguments) method object
arguments
help decode S
S.capitalize()
S.ljust(width [, fill])
S.center(width [, fill])
S.lower()
S.count(sub [, start [, end]])
S.lstrip([chars])
S.encode([encoding [,errors]])
S.maketrans(x[, y[, z]])
S.endswith(suffix [, start [, end]])
S.partition(sep)
S.expandtabs([tabsize])
S.replace(old, new [, count])
S.find(sub [, start [, end]])
S.rfind(sub [,start [,end]])
S.format(fmtstr, *args, **kwargs)
S.rindex(sub [, start [, end]])
S.index(sub [, start [, end]])
S.rjust(width [, fill])
S.isalnum()
S.rpartition(sep)
S.isalpha()
S.rsplit([sep[, maxsplit]])
S.isdecimal()
S.rstrip([chars])
S.isdigit()
S.split([sep [,maxsplit]])
String Methods | 173
www.it-ebooks.info
S.isidentifier()
S.splitlines([keepends])
S.islower()
S.startswith(prefix [, start [, end]])
S.isnumeric()
S.strip([chars])
S.isprintable()
S.swapcase()
S.isspace()
S.title()
S.istitle()
S.translate(map)
S.isupper()
S.upper()
S.join(iterable)
S.zfill(width)
String Method Examples: Changing Strings
>>> S = 'spammy' >>> S = S[:3] + 'xx' + S[5:] >>> S 'spaxxy'
replace >>> S = 'spammy' >>> S = S.replace('mm', 'xx') >>> S 'spaxxy'
replace
>>> 'aa$bb$cc$dd'.replace('$', 'SPAM') 'aaSPAMbbSPAMccSPAMdd'
replace
174 | Chapter 7: Strings
www.it-ebooks.info
find >>> S = 'xxxxSPAMxxxxSPAMxxxx' >>> where = S.find('SPAM') >>> where 4 >>> S = S[:where] + 'EGGS' + S[(where+4):] >>> S 'xxxxEGGSxxxxSPAMxxxx'
find −1 in
find replace
>>> S = 'xxxxSPAMxxxxSPAMxxxx' >>> S.replace('SPAM', 'EGGS') 'xxxxEGGSxxxxEGGSxxxx' >>> S.replace('SPAM', 'EGGS', 1) 'xxxxEGGSxxxxSPAMxxxx'
replace
replace
>>> S = 'spammy' >>> L = list(S) >>> L ['s', 'p', 'a', 'm', 'm', 'y']
list
>>> L[3] = 'x' >>> L[4] = 'x' >>> L ['s', 'p', 'a', 'x', 'x', 'y']
join
String Methods | 175
www.it-ebooks.info
>>> S = ''.join(L) >>> S 'spaxxy'
join join
>>> 'SPAM'.join(['eggs', 'sausage', 'ham', 'toast']) 'eggsSPAMsausageSPAMhamSPAMtoast'
bytearray list join
String Method Examples: Parsing Text
>>> line = 'aaa bbb ccc' >>> col1 = line[0:3] >>> col3 = line[8:] >>> col1 'aaa' >>> col3 'ccc'
>>> line = 'aaa bbb ccc' >>> cols = line.split() >>> cols ['aaa', 'bbb', 'ccc']
split
176 | Chapter 7: Strings
www.it-ebooks.info
>>> line = 'bob,hacker,40' >>> line.split(',') ['bob', 'hacker', '40']
>>> line = "i'mSPAMaSPAMlumberjack" >>> line.split("SPAM") ["i'm", 'a', 'lumberjack']
Other Common String Methods in Action
>>> line = "The knights who say Ni!\n" >>> line.rstrip() 'The knights who say Ni!' >>> line.upper() 'THE KNIGHTS WHO SAY NI!\n' >>> line.isalpha() False >>> line.endswith('Ni!\n') True >>> line.startswith('The') True
in endswith >>> line 'The knights who say Ni!\n' >>> line.find('Ni') != −1 True >>> 'Ni' in line True >>> sub = 'Ni!\n' >>> line.endswith(sub) True >>> line[-len(sub):] == sub True
format
String Methods | 177
www.it-ebooks.info
help(S.method)
method
S
re
re
The Original string Module (Gone in 3.0) string
string string
string X X.method(arguments)
string string.method(X, arguments)
>>> S = 'a+b+c+' >>> x = S.replace('+', 'spam') >>> x 'aspambspamcspam'
string >>> import string >>> y = string.replace(S, '+', 'spam') >>> y 'aspambspamcspam'
178 | Chapter 7: Strings
www.it-ebooks.info
string import from string
String Formatting Expressions
% %
String Formatting Expressions | 179
www.it-ebooks.info
%
% %
%d
%
%d
'dead'
>>> 'That is %d %s bird!' % (1, 'dead') That is 1 dead bird!
>>> exclamation = "Ni" >>> "The knights who say %s!" % exclamation 'The knights who say Ni!' >>> "%d %s %d you" % (1, 'spam', 4) '1 spam 4 you' >>> "%s -- %s -- %s" % (42, 3.14159, [1, 2, 3]) '42 -- 3.14159 -- [1, 2, 3]'
"Ni" %s %
%s %s %s
180 | Chapter 7: Strings
1 %s
www.it-ebooks.info
Advanced String Formatting Expressions % printf printf %e %f
Code
Meaning
s
String (or any object’s str(X) string)
r
s, but uses repr, not str
c
Character
d
Decimal (integer)
i
Integer
u
Same as d (obsolete: no longer unsigned)
o
Octal integer
x
Hex integer
X
x, but prints uppercase
e
Floating-point exponent, lowercase
E
Same as e, but prints uppercase
f
Floating-point decimal
F
Floating-point decimal
g
Floating-point e or f
G
Floating-point E or F
%
Literal %
%g
%[(name)][flags][width][.precision]typecode
% −
+
0 width
precision
*
String Formatting Expressions | 181
www.it-ebooks.info
>>> x = 1234 >>> res = "integers: ...%d...%−6d...%06d" % (x, x, x) >>> res 'integers: ...1234...1234 ...001234'
%e %f
%g %E
%e
>>> x = 1.23456789 >>> x 1.2345678899999999 >>> '%e | %f | %g' % (x, x, x) '1.234568e+00 | 1.234568 | 1.23457' >>> '%E' % x '1.234568E+00'
str >>> '%−6.2f | %05.2f | %+06.1f' % (x, x, x) '1.23 | 01.23 | +001.2' >>> "%s" % x, str(x) ('1.23456789', '1.23456789')
* % >>> '%f, %.2f, %.*f' % (1/3.0, 1/3.0, 4, 1/3.0) '0.333333, 0.33, 0.3333'
Dictionary-Based String Formatting Expressions
>>> "%(n)d %(x)s" % {"n":1, "x":"spam"} '1 spam'
182 | Chapter 7: Strings
www.it-ebooks.info
(n)
(x)
>>> reply = """ Greetings... Hello %(name)s! Your age squared is %(age)s """ >>> values = {'name': 'Bob', 'age': 40} >>> print(reply % values) Greetings... Hello Bob! Your age squared is 40
vars >>> food = 'spam' >>> age = 40 >>> vars() {'food': 'spam', 'age': 40, ...many more... }
>>> "%(age)d %(food)s" % vars() '40 spam'
%x
%o
String Formatting Method Calls
String Formatting Method Calls | 183
www.it-ebooks.info
The Basics format
{1} {food}
>>> template = '{0}, {1} and {2}' >>> template.format('spam', 'ham', 'eggs') 'spam, ham and eggs' >>> template = '{motto}, {pork} and {food}' >>> template.format(motto='spam', pork='ham', food='eggs') 'spam, ham and eggs' >>> template = '{motto}, {0} and {food}' >>> template.format('ham', motto='spam', food='eggs') 'spam, ham and eggs'
>>> '{motto}, {0} and {food}'.format(42, motto=3.14, food=[1, 2]) '3.14, 42 and [1, 2]'
%
format format
>>> X = '{motto}, {0} and {food}'.format(42, motto=3.14, food=[1, 2]) >>> X '3.14, 42 and [1, 2]' >>> X.split(' and ') ['3.14, 42', '[1, 2]'] >>> Y = X.replace('and', 'but under no circumstances') >>> Y '3.14, 42 but under no circumstances [1, 2]'
Adding Keys, Attributes, and Offsets %
184 | Chapter 7: Strings
www.it-ebooks.info
sys >>> import sys >>> 'My {1[spam]} runs {0.platform}'.format(sys, {'spam': 'laptop'}) 'My laptop runs win32' >>> 'My {config[spam]} runs {sys.platform}'.format(sys=sys, config={'spam': 'laptop'}) 'My laptop runs win32'
%
>>> somelist = list('SPAM') >>> somelist ['S', 'P', 'A', 'M'] >>> 'first={0[0]}, third={0[2]}'.format(somelist) 'first=S, third=A' >>> 'first={0}, last={1}'.format(somelist[0], somelist[-1]) 'first=S, last=M' >>> parts = somelist[0], somelist[-1], somelist[1:3] >>> 'first={0}, last={1}, middle={2}'.format(*parts) "first=S, last=M, middle=['P', 'A']"
Adding Specific Formatting %
{fieldname!conversionflag:formatspec}
fieldname conversionflag
r s
a
repr str
ascii
String Formatting Method Calls | 185
www.it-ebooks.info
formatspec
formatspec [[fill]align][sign][#][0][width][.precision][typecode]
align
< > =
^ formatspec
{} * % bin
{0:10} {1:10}
platform
>>> '{0:10} = {1:10}'.format('spam', 123.4567) 'spam = 123.457' >>> '{0:>10} = {1:10} = {1[item]:>> T[1][0] = 'spam' >>> T (1, ['spam', 3], 4)
Why Lists and Tuples?
Files
Files | 229
www.it-ebooks.info
open open
Operation
Interpretation
output = open(r'C:\spam', 'w')
Create output file ('w' means write)
input = open('data', 'r')
Create input file ('r' means read)
input = open('data')
Same as prior line ('r' is the default)
aString = input.read()
Read entire file into a single string
aString = input.read(N)
Read up to next N characters (or bytes) into a string
aString = input.readline()
Read next line (including \n newline) into a string
aList = input.readlines()
Read entire file into list of line strings (with \n)
output.write(aString)
Write a string of characters (or bytes) into file
output.writelines(aList)
Write all line strings in a list into file
output.close()
Manual close (done for you when file is collected)
output.flush()
Flush output buffer to disk without closing
anyFile.seek(N)
Change file position to offset N for next operation
for line in open('data'): use line
File iterators read line by line
open('f.txt', encoding='latin-1')
Python 3.0 Unicode text files (str strings)
open('f.bin', 'rb')
Python 3.0 binary bytes files (bytes strings)
Opening Files open 'w'
b
230 | Chapter 9: Tuples, Files, and Everything Else
'r' 'a'
www.it-ebooks.info
+
open
Using Files open
for
print
int float str pickle struct close close
Files | 231
www.it-ebooks.info
close
close
with as
flush open seek
Files in Action \n readline
>>> >>> 16 >>> 18 >>>
readline
myfile = open('myfile.txt', 'w') myfile.write('hello text file\n') myfile.write('goodbye text file\n') myfile.close()
>>> myfile = open('myfile.txt') >>> myfile.readline() 'hello text file\n' >>> myfile.readline() 'goodbye text file\n' >>> myfile.readline() ''
write \n
232 | Chapter 9: Tuples, Files, and Everything Else
www.it-ebooks.info
read >>> open('myfile.txt').read() 'hello text file\ngoodbye text file\n' >>> print(open('myfile.txt').read()) hello text file goodbye text file
>>> for line in open('myfile'): ... print(line, end='') ... hello text file goodbye text file
open
Text and binary files in Python 3.0 open
str bytes
unicode
codecs.open
Files | 233
www.it-ebooks.info
bytes
str
bytes
>>> data = open('data.bin', 'rb').read() >>> data b'\x00\x00\x00\x07spam\x00\x08' >>> data[4:8] b'spam' >>> data[0] 115 >>> bin(data[0]) '0b1110011'
\n
Storing and parsing Python objects in files
write >>> >>> >>> >>> >>> >>> >>> >>> >>> >>>
X, Y, Z = 43, 44, 45 S = 'Spam' D = {'a': 1, 'b': 2} L = [1, 2, 3] F = open('datafile.txt', 'w') F.write(S + '\n') F.write('%s,%s,%s\n' % (X, Y, Z)) F.write(str(L) + '$' + str(D) + '\n') F.close()
print >>> chars = open('datafile.txt').read() >>> chars
234 | Chapter 9: Tuples, Files, and Everything Else
www.it-ebooks.info
"Spam\n43,44,45\n[1, 2, 3]${'a': 1, 'b': 2}\n" >>> print(chars) Spam 43,44,45 [1, 2, 3]${'a': 1, 'b': 2}
>>> F = open('datafile.txt') >>> line = F.readline() >>> line 'Spam\n' >>> line.rstrip() 'Spam'
rstrip line[:−1] \n
>>> line = F.readline() >>> line '43,44,45\n' >>> parts = line.split(',') >>> parts ['43', '44', '45\n']
split
>>> int(parts[1]) 44 >>> numbers = [int(P) for P in parts] >>> numbers [43, 44, 45]
int
rstrip
\n
int
eval >>> line = F.readline() >>> line
Files | 235
www.it-ebooks.info
"[1, 2, 3]${'a': 1, 'b': 2}\n" >>> parts = line.split('$') >>> parts ['[1, 2, 3]', "{'a': 1, 'b': 2}\n"] >>> eval(parts[0]) [1, 2, 3] >>> objects = [eval(P) for P in parts] >>> objects [[1, 2, 3], {'a': 1, 'b': 2}]
Storing native Python objects with pickle eval eval
pickle pickle
>>> >>> >>> >>> >>>
D = {'a': 1, 'b': 2} F = open('datafile.pkl', 'wb') import pickle pickle.dump(D, F) F.close()
pickle >>> F = open('datafile.pkl', 'rb') >>> E = pickle.load(F) >>> E {'a': 1, 'b': 2}
pickle pickle >>> open('datafile.pkl', 'rb').read() b'\x80\x03}q\x00(X\x01\x00\x00\x00aq\x01K\x01X\x01\x00\x00\x00bq\x02K\x02u.'
pickle pickle shelve
pickle help shelve
236 | Chapter 9: Tuples, Files, and Everything Else
pickle
www.it-ebooks.info
shelve
pickle
bytes str
bytes cPickle pickle _pickle pickle
pickle
Storing and parsing packed binary data in files
struct
'wb' struct
>>> F = open('data.bin', 'wb') >>> import struct >>> data = struct.pack('>i4sh', 7, 'spam', 8) >>> data b'\x00\x00\x00\x07spam\x00\x08' >>> F.write(data) >>> F.close()
bytes
>>> F = open('data.bin', 'rb') >>> data = F.read() >>> data b'\x00\x00\x00\x07spam\x00\x08'
Files | 237
www.it-ebooks.info
>>> values = struct.unpack('>i4sh', data) >>> values (7, 'spam', 8)
struct 'wb'
help 'rb'
File context managers
with open(r'C:\misc\data.txt') as myfile: for line in myfile: ...use line here...
try/finally
myfile = open(r'C:\misc\data.txt') try: for line in myfile: ...use line here... finally: myfile.close()
Other File Tools seek flush
dir file
238 | Chapter 9: Tuples, Files, and Everything Else
help
www.it-ebooks.info
open
sys
sys.stdout
os
os.popen
subprocess.Popen
open file open
file
open file
open file
io type(F)
F
Type Categories Revisited
Type Categories Revisited | 239
www.it-ebooks.info
str
bytes
bytearray
frozenset
set
Object type
Category
Mutable?
Numbers (all)
Numeric
No
Strings
Sequence
No
Lists
Sequence
Yes
Dictionaries
Mapping
Yes
Tuples
Sequence
No
Files
Extension
N/A
Sets
Set
Yes
frozenset
Set
No
bytearray (3.0)
Sequence
Yes
Why You Will Care: Operator Overloading
class MySequence: def __getitem__(self, index): # Called on self[index], others def __add__(self, other): # Called on self + other
240 | Chapter 9: Tuples, Files, and Everything Else
unicode
www.it-ebooks.info
__setitem__ self[index]=value
Object Flexibility
>>> L = ['abc', [(1, 2), ([3], 4)], 5] >>> L[1] [(1, 2), ([3], 4)] >>> L[1][1] ([3], 4) >>> L[1][1][0] [3] >>> L[1][1][0][0] 3
References Versus Copies
References Versus Copies | 241
www.it-ebooks.info
X
L D
X X
>>> X = [1, 2, 3] >>> L = ['a', X, 'b'] >>> D = {'x':X, 'y':2}
X L
>>> X[1] = 'surprise' >>> L ['a', [1, 'surprise', 3], 'b'] >>> D {'x': [1, 'surprise', 3], 'y': 2}
242 | Chapter 9: Tuples, Files, and Everything Else
D
www.it-ebooks.info
L[:] copy
X.copy() list
list(L)
copy
>>> L = [1,2,3] >>> D = {'a':1, 'b':2}
>>> A = L[:] >>> B = D.copy()
>>> A[1] = 'Ni' >>> B['c'] = 'spam' >>> >>> L, D ([1, 2, 3], {'a': 1, 'b': 2}) >>> A, B ([1, 'Ni', 3], {'a': 1, 'c': 'spam', 'b': 2})
References Versus Copies | 243
www.it-ebooks.info
>>> X = [1, 2, 3] >>> L = ['a', X[:], 'b'] >>> D = {'x':X[:], 'y':2}
L X
D X
L
D
X
L
D
X copy
copy
import copy Y
copy(Y)
Comparisons, Equality, and Truth
>>> L1 >>> L2 >>> L1 (True,
= [1, ('a', 3)] = [1, ('a', 3)] == L2, L1 is L2 False)
L1
L2
== is L1
L2
== is
>>> S1 = 'spam' >>> S2 = 'spam'
244 | Chapter 9: Tuples, Files, and Everything Else
X = copy.deep
www.it-ebooks.info
>>> S1 == S2, S1 is S2 (True, True)
==
is 'spam' S1 >>> S1 >>> S2 >>> S1 (True,
S2
is
= 'a longer string' = 'a longer string' == S2, S1 is S2 False)
== is
>>> L1 = [1, ('a', 3)] >>> L2 = [1, ('a', 2)] >>> L1 < L2, L1 == L2, L1 > L2 (False, False, True)
L1
L2
3
2
"abc" < "ac" (key, value) (key, value) 1 < 'spam'
Comparisons, Equality, and Truth | 245
www.it-ebooks.info
Python 3.0 Dictionary Comparisons
C:\misc> c:\python26\python >>> D1 = {'a':1, 'b':2} >>> D2 = {'a':1, 'b':3} >>> D1 == D2 False >>> D1 < D2 True
items
sorted
C:\misc> c:\python30\python >>> D1 = {'a':1, 'b':2} >>> D2 = {'a':1, 'b':3} >>> D1 == D2 False >>> D1 < D2 TypeError: unorderable types: dict() < dict() >>> list(D1.items()) [('a', 1), ('b', 2)] >>> sorted(D1.items()) [('a', 1), ('b', 2)] >>> sorted(D1.items()) < sorted(D2.items()) True >>> sorted(D1.items()) > sorted(D2.items()) False
The Meaning of True and False in Python True
False 0
1
246 | Chapter 9: Tuples, Files, and Everything Else
www.it-ebooks.info
Object
Value
"spam"
True
""
False
[]
False
{}
False
1
True
0.0
False
None
False
if X:
X
if X != '': if
The None object None
None NULL
None >>> L = [None] * 100 >>> >>> L [None, None, None, None, None, None, None, ... ]
None
Comparisons, Equality, and Truth | 247
www.it-ebooks.info
None
None
The bool type bool True
False
1 1
True 1
0
0
False
0 True 1
False
0 if bool
>>> bool(1) True >>> bool('spam') True >>> bool({}) False
if
Python’s Type Hierarchies
248 | Chapter 9: Tuples, Files, and Everything Else
www.it-ebooks.info
Python’s Type Hierarchies | 249
www.it-ebooks.info
Type Objects type type(X)
X if
dict list str tuple int float complex bytes type set file open
types
isinstance type([1]) == type([]) type([1]) == list isinstance([1], list) import types def f(): pass type(f) == types.FunctionType
isinstance
type(X)
__class__
Other Types in Python
250 | Chapter 9: Tuples, Files, and Everything Else
www.it-ebooks.info
4 [1,2]
open
def
lambda re
re.compile()
Built-in Type Gotchas
Assignment Creates References, Not Copies L M
L
L M
>>> L = [1, 2, 3] >>> M = ['X', L, 'Y'] >>> M ['X', [1, 2, 3], 'Y'] >>> L[1] = 0 >>> M ['X', [1, 0, 3], 'Y']
>>> L = [1, 2, 3] >>> M = ['X', L[:], 'Y'] >>> L[1] = 0 >>> L [1, 0, 3] >>> M ['X', [1, 2, 3], 'Y']
Built-in Type Gotchas | 251
www.it-ebooks.info
Repetition Adds One Level Deep X
L
Y
L >>> L = [4, 5, 6] >>> X = L * 4 >>> Y = [L] * 4 >>> X [4, 5, 6, 4, 5, 6, 4, 5, 6, 4, 5, 6] >>> Y [[4, 5, 6], [4, 5, 6], [4, 5, 6], [4, 5, 6]]
L
Y L
>>> L[1] = 0 >>> X [4, 5, 6, 4, 5, 6, 4, 5, 6, 4, 5, 6] >>> Y [[4, 0, 6], [4, 0, 6], [4, 0, 6], [4, 0, 6]]
Beware of Cyclic Data Structures [...] >>> L = ['grail'] >>> L.append(L) >>> L ['grail', [...]]
252 | Chapter 9: Tuples, Files, and Everything Else
www.it-ebooks.info
Immutable Types Can’t Be Changed In-Place
T = (1, 2, 3) T[2] = 4 T = T[:2] + (4,)
Chapter Summary open pickle
struct
Chapter Summary | 253
www.it-ebooks.info
Test Your Knowledge: Quiz (4, 5, 6) (1, 5, 6) open
Test Your Knowledge: Answers len
index
T = (4, 5, 6) T = (1,) + T[1:]
open pickle struct copy
copy.deepcopy(X) X aList[:] aDict.copy()
254 | Chapter 9: Tuples, Files, and Everything Else
'r'
www.it-ebooks.info
1
True 0
False
Test Your Knowledge: Part II Exercises
X=1;X X,Y,Z 2 ** 16 2 / 5, 2 / 5.0 "spam" + "eggs" S = "ham" "eggs " + S S * 5 S[:0] "green %s and %s" % ("eggs", S) 'green {0} and {1}'.format('eggs', S) ('x',)[0] ('x', 'y')[1] L = [1,2,3] + [4,5,6] L, L[:], L[:0], L[−2], L[−2:] ([1,2,3] + [4,5,6])[2:4] [L[2], L[3]] L.reverse(); L L.sort(); L L.index(4) {'a':1, 'b':2}['b'] D = {'x':1, 'y':2, 'z':3}
Test Your Knowledge: Part II Exercises | 255
www.it-ebooks.info
D['w'] = 0 D['x'] + D['w'] D[(1,2,3)] = 4 list(D.keys()), list(D.values()), (1,2,3) in D [[]], ["",[],(),{},None]
L L=[0,1,2,3]
L[4] L[−1000:100] L[3:1] L[3:1]=['?'] del
L L[2]=[]
L[2:3]=[] del del L[0] del L[1:]
L[1:2]=1
>>> X = 'spam' >>> Y = 'eggs' >>> X, Y = Y, X
X >>> D = {} >>> D[1] = 'a' >>> D[2] = 'b'
>>> D[(1, 2, 3)] = 'c' >>> D {1: 'a', 2: 'b', (1, 2, 3): 'c'}
256 | Chapter 9: Tuples, Files, and Everything Else
Y
www.it-ebooks.info
D 'b'
'a' D['d'] D['d']='spam'
'c' 'd'
+ +
+
+ append keys
append
S S[0][0][0][0][0]
S = "spam"
['s', 'p', 'a', 'm'] S
S = "spam" "slam"
"Hello file world!"
open write \n
Test Your Knowledge: Part II Exercises | 257
www.it-ebooks.info
www.it-ebooks.info
PART III
Statements and Syntax
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 10
Introducing Python Statements
Python Program Structure Revisited
261
www.it-ebooks.info
Python’s Statements break
continue
del
Statement
Role
Example
Assignment
Creating references
a, *b = 'good', 'bad', 'ugly'
Calls and other expressions
Running functions
log.write("spam, ham")
print calls
Printing objects
print('The Killer', joke)
if/elif/else
Selecting actions
if "python" in text: print(text)
for/else
Sequence iteration
for x in mylist: print(x)
while/else
General loops
while X > Y: print('hello')
pass
Empty placeholder
while True: pass
break
Loop exit
while True: if exittest(): break
continue
Loop continue
while True: if skiptest(): continue
def
Functions and methods
def f(a, b, c=1, *d): print(a+b+c+d[0])
return
Functions results
def f(a, b, c=1, *d): return a+b+c+d[0]
yield
Generator functions
def gen(n): for i in n: yield i*2
global
Namespaces
x = 'old' def function(): global x, y; x = 'new'
nonlocal
Namespaces (3.0+)
def outer(): x = 'old' def function(): nonlocal x; x = 'new'
import
Module access
import sys
from
Attribute access
from sys import stdin
class
Building objects
class Subclass(Superclass): staticData = [] def method(self): pass
262 | Chapter 10: Introducing Python Statements
www.it-ebooks.info
Statement
Role
Example
try/except/ finally
Catching exceptions
try:
raise
Triggering exceptions
raise EndSearch(location)
assert
Debugging checks
assert X > Y, 'X too small'
with/as
Context managers (2.6+)
with open('data') as myfile: process(myfile)
del
Deleting references
del del del del
action() except: print('action error')
data[k] data[i:j] obj.attr variable
print
yield
print
yield
print
nonlocal print exec
try except
try finally except
finally
try with as from __future__ import with_statement
Python Program Structure Revisited | 263
www.it-ebooks.info
A Tale of Two ifs
if if (x > y) { x = 1; y = 2; }
if x > y: x = 1 y = 2
What Python Adds :
Header line: Nested statement block
264 | Chapter 10: Introducing Python Statements
www.it-ebooks.info
What Python Removes
Parentheses are optional if (x < y)
if x < y
End of line is end of statement
x = 1;
x = 1
A Tale of Two ifs | 265
www.it-ebooks.info
End of indentation is end of block
begin end then endif if (x > y) { x = 1; y = 2; }
if x > y: x = 1 y = 2
Why Indentation Syntax?
266 | Chapter 10: Introducing Python Statements
www.it-ebooks.info
while while (x > 0) {
while (x > 0) { --------; --------;
while (x > 0) { --------; --------; --------; --------;
while (x > 0) { --------; --------; --------; --------; --------; --------; }
}
if (x) if (y) statement1; else statement2;
A Tale of Two ifs | 267
www.it-ebooks.info
if
else if (y)
if
else
if (x)
if x:
if y:
statement1
else:
statement2
if
else if x
268 | Chapter 10: Introducing Python Statements
www.it-ebooks.info
A Few Special Cases
Statement rule special cases
a = 1; b = 2; print(a + b)
print
()
[]
{}
mlist = [111, 222, 333]
A Tale of Two ifs | 269
www.it-ebooks.info
X = (A + B + C + D)
if (A == 1 and B == 2 and C == 3): print('spam' * 3)
X = A + B + \ C + D
Block rule special case
if x > y: print(x)
if
print else
if
270 | Chapter 10: Introducing Python Statements
www.it-ebooks.info
if
A Quick Example: Interactive Loops
A Simple Interactive Loop
while True: reply = input('Enter text:') if reply == 'stop': break print(reply.upper())
while while while True input
if if
A Quick Example: Interactive Loops | 271
www.it-ebooks.info
break while
while while
Enter text:spam SPAM Enter text:42 42 Enter text:stop
raw_input print
input print
Doing Math on User Inputs
>>> reply = '20' >>> reply ** 2 ...error text omitted... TypeError: unsupported operand type(s) for ** or pow(): 'str' and 'int'
272 | Chapter 10: Introducing Python Statements
www.it-ebooks.info
>>> int(reply) ** 2 400
while True: reply = input('Enter text:') if reply == 'stop': break print(int(reply) ** 2) print('Bye')
if print
Enter text:2 4 Enter text:40 1600 Enter text:stop Bye
print
print
Handling Errors by Testing Inputs Enter text:xxx ...error text omitted... ValueError: invalid literal for int() with base 10: 'xxx'
int isdigit >>> S = '123' >>> T = 'xxx' >>> S.isdigit(), T.isdigit() (True, False)
if while True: reply = input('Enter text:')
A Quick Example: Interactive Loops | 273
www.it-ebooks.info
if reply == 'stop': break elif not reply.isdigit(): print('Bad!' * 8) else: print(int(reply) ** 2) print('Bye')
if if elif else else if elif
else
if
if if
Enter text:5 25 Enter text:xyz Bad!Bad!Bad!Bad!Bad!Bad!Bad!Bad! Enter text:10 100 Enter text:stop
Handling Errors with try Statements try try while True: reply = input('Enter text:') if reply == 'stop': break try: num = int(reply) except: print('Bad!' * 8) else: print(int(reply) ** 2) print('Bye')
274 | Chapter 10: Introducing Python Statements
print while
www.it-ebooks.info
try
try
except
else try
try
except
else try except else if
try
else try
if try
else try else
try else
try
if
break try try
try float
Nesting Code Three Levels Deep
while True: reply = input('Enter text:') if reply == 'stop': break elif not reply.isdigit(): print('Bad!' * 8) else: num = int(reply) if num < 20: print('low') else: print(num ** 2) print('Bye')
A Quick Example: Interactive Loops | 275
www.it-ebooks.info
if
else while
Enter text:19 low Enter text:20 400 Enter text:spam Bad!Bad!Bad!Bad!Bad!Bad!Bad!Bad! Enter text:stop Bye
Chapter Summary
Test Your Knowledge: Quiz
try
276 | Chapter 10: Introducing Python Statements
if
www.it-ebooks.info
Test Your Knowledge: Answers
try
Test Your Knowledge: Answers | 277
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 11
Assignments, Expressions, and Prints
Assignment Statements
279
www.it-ebooks.info
= for
Assignment Statement Forms
Operation
Interpretation
spam = 'Spam'
Basic form
spam, ham = 'yum', 'YUM'
Tuple assignment (positional)
[spam, ham] = ['yum', 'YUM']
List assignment (positional)
a, b, c, d = 'spam'
Sequence assignment, generalized
a, *b = 'spam'
Extended sequence unpacking (Python 3.0)
spam = ham = 'lunch'
Multiple-target assignment
spams += 42
Augmented assignment (equivalent to spams = spams + 42)
= spam
'yum'
'YUM'
a
's' b
280 | Chapter 11: Assignments, Expressions, and Prints
'p'
ham
www.it-ebooks.info
a
a 's'
b b
'pam'
spam ham = 'lunch'
'lunch' spam = ham
ham
ham
spam += 42
spam = spam + 42
Sequence Assignments % python >>> nudge = 1 >>> wink = 2 >>> A, B = nudge, wink >>> A, B (1, 2) >>> [C, D] = [nudge, wink] >>> C, D (1, 2)
Assignment Statements | 281
www.it-ebooks.info
>>> >>> >>> >>> (2,
nudge = 1 wink = 2 nudge, wink = wink, nudge nudge, wink 1)
>>> [a, b, c] = (1, 2, 3) >>> a, c (1, 3) >>> (a, b, c) = "ABC" >>> a, c ('A', 'C')
Advanced sequence assignment patterns =
>>> string = 'SPAM' >>> a, b, c, d = string >>> a, d ('S', 'M') >>> a, b, c = string ...error text omitted... ValueError: too many values to unpack
>>> a, b, c = string[0], string[1], string[2:] >>> a, b, c ('S', 'P', 'AM') >>> a, b, c = list(string[:2]) + [string[2:]] >>> a, b, c
282 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
('S', 'P', 'AM') >>> a, b = string[:2] >>> c = string[2:] >>> a, b, c ('S', 'P', 'AM') >>> (a, b), c = string[:2], string[2:] >>> a, b, c ('S', 'P', 'AM')
>>> ((a, b), c) = ('SP', 'AM') >>> a, b, c ('S', 'P', 'AM')
'SP'
(a, b) 'AM'
c
for for (a, b, c) in [(1, 2, 3), (4, 5, 6)]: ... for ((a, b), c) in [((1, 2), 3), ((4, 5), 6)]: ...
def f(((a, b), c)): f(((1, 2), 3))
>>> red, green, blue = range(3) >>> red, blue (0, 2)
0 1
2
range
Assignment Statements | 283
www.it-ebooks.info
>>> range(3) [0, 1, 2]
range
for
>>> L = [1, 2, 3, 4] >>> while L: ... front, L = L[0], L[1:] ... print(front, L) ... 1 [2, 3, 4] 2 [3, 4] 3 [4] 4 []
... ...
front = L[0] L = L[1:]
append
pop
front =
L.pop(0) while for
Extended Sequence Unpacking in Python 3.0 *X
Extended unpacking in action
C:\misc> c:\python30\python >>> seq = [1, 2, 3, 4] >>> a, b, c, d = seq >>> print(a, b, c, d) 1 2 3 4
284 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
>>> a, b = seq ValueError: too many values to unpack
a b >>> >>> 1 >>> [2,
a, *b = seq a b 3, 4]
b a >>> >>> [1, >>> 4
*a, b = seq a 2, 3] b
a
c
b >>> >>> 1 >>> [2, >>> 4
a, *b, c = seq a
>>> >>> 1 >>> 2 >>> [3,
a, b, *c = seq a
b 3] c
b c 4]
>>> a, *b = 'spam' >>> a, b ('s', ['p', 'a', 'm']) >>> a, *b, c = 'spam'
Assignment Statements | 285
www.it-ebooks.info
>>> a, b, c ('s', ['p', 'a'], 'm')
>>> S = 'spam' >>> S[0], S[1:] ('s', 'pam') >>> S[0], S[1:3], S[3] ('s', 'pa', 'm')
>>> L = [1, 2, 3, 4] >>> while L: ... front, *L = L ... print(front, L) ... 1 [2, 3, 4] 2 [3, 4] 3 [4] 4 []
Boundary cases
>>> seq [1, 2, 3, 4] >>> a, b, c, *d = seq >>> print(a, b, c, d) 1 2 3 [4]
a b c e >>> a, b, c, d, *e = seq >>> print(a, b, c, d, e) 1 2 3 4 [] >>> a, b, *e, c, d = seq >>> print(a, b, c, d, e) 1 2 3 4 []
286 | Chapter 11: Assignments, Expressions, and Prints
d
www.it-ebooks.info
>>> a, *b, c, *d = seq SyntaxError: two starred expressions in assignment >>> a, b = seq ValueError: too many values to unpack >>> *a = seq SyntaxError: starred assignment target must be in a list or tuple >>> *a, = seq >>> a [1, 2, 3, 4]
A useful convenience
>>> seq [1, 2, 3, 4] >>> a, *b = seq >>> a, b (1, [2, 3, 4]) >>> a, b = seq[0], seq[1:] >>> a, b (1, [2, 3, 4])
>>> *a, b = seq >>> a, b ([1, 2, 3], 4) >>> a, b = seq[:-1], seq[-1] >>> a, b ([1, 2, 3], 4)
Assignment Statements | 287
www.it-ebooks.info
Application to for loops for for for for (a, *b, c) in [(1, 2, 3, 4), (5, 6, 7, 8)]: ...
a, *b, c = (1, 2, 3, 4)
a b
# b gets [2, 3]
c
for (a, b, c) in [(1, 2, 3), (4, 5, 6)]:
# a, b, c = (1, 2, 3), ...
for all in [(1, 2, 3, 4), (5, 6, 7, 8)]: a, b, c = all[0], all[1:3], all[3]
for
Multiple-Target Assignments a b 'spam' >>> a = b = c = 'spam' >>> a, b, c ('spam', 'spam', 'spam')
>>> c = 'spam' >>> b = c >>> a = b
Multiple-target assignment and shared references
288 | Chapter 11: Assignments, Expressions, and Prints
c
www.it-ebooks.info
>>> >>> >>> (0,
a = b = 0 b = b + 1 a, b 1)
b
b
>>> a = b = [] >>> b.append(42) >>> a, b ([42], [42])
a
b
b
a
>>> a = [] >>> b = [] >>> b.append(42) >>> a, b ([], [42])
Augmented Assignments
X = X + Y X += Y
X += Y
X &= Y
X -= Y
X |= Y
X *= Y
X ^= Y
X /= Y
X >>= Y
X %= Y
X > >>> >>> 2 >>> >>> 3
x = 1 x = x + 1 x x += 1 x
S = S + "SPAM" >>> S = "spam" >>> S += "SPAM" >>> S 'spamSPAM'
X *= Y
X >>= Y
X //= Y
X += Y X X = X + Y X
append >>> >>> >>> [1, >>> >>> [1,
L = [1, 2] L = L + [3] L 2, 3] L.append(4) L 2, 3, 4]
X += Y X++ −−X
290 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
extend >>> >>> [1, >>> >>> [1,
L = L + [5, 6] L 2, 3, 4, 5, 6] L.extend([7, 8]) L 2, 3, 4, 5, 6, 7, 8]
extend + >>> L += [9, 10] >>> L [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Augmented assignment and shared references += +
>>> L = [1, 2] >>> M = L >>> L = L + [3, 4] >>> L, M ([1, 2, 3, 4], [1, 2]) >>> L = [1, 2] >>> M = L >>> L += [3, 4] >>> L, M ([1, 2, 3, 4], [1, 2, 3, 4])
L[len(L):] = [11,12,13] extend
Assignment Statements | 291
www.it-ebooks.info
Variable Name Rules
_spam spam 1_Spam spam$
Spam_1
@#!
SPAM
spam X
x
class klass
finally
is
Class
False
class
return
None
continue
for
lambda
try
True
def
from
nonlocal
while
and
del
global
not
with
as
elif
if
or
yield
assert
else
import
pass
break
except
in
raise
print exec nonlocal
292 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
with
as
yield yield
and = 1 True False None
import
Python’s Deprecation Protocol
yield yield
Assignment Statements | 293
www.it-ebooks.info
with
as
from__future__
with
as
Naming conventions
__name__
_X
from module
import * __X__ __X
_
self open = 42
Names have no type, but objects do
294 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
>>> x = 0 >>> x = "Hello" >>> x = [1, 2, 3]
Expression Statements
const _X
__X
Expression Statements | 295
www.it-ebooks.info
print
Operation
Interpretation
spam(eggs, ham)
Function calls
spam.ham(eggs)
Method calls
spam
Printing variables in the interactive interpreter
print(a, b, c, sep='')
Printing operations in Python 3.0
yield x ** 2
Yielding expression statements
yield print None >>> x = print('spam') spam >>> print(x) None
= = == while
Expression Statements and In-Place Changes >>> >>> >>> [1,
L = [1, 2] L.append(3) L 2, 3]
296 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
L >>> L = L.append(4) >>> print(L) None
append sort
reverse None
Print Operations print
print
file.write(str) print
stdout
stdout
sys
sys.stdout print
stdout print
Print Operations | 297
www.it-ebooks.info
The Python 3.0 print Function print None
print
Call format print print([object, ...][, sep=' '][, end='\n'][, file=sys.stdout])
= object end sep end
sep
file
file
print
298 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
sep end
\n print
file sys.stdout write(string) object str print
The 3.0 print function in action
C:\misc> c:\python30\python >>> >>> print() >>> x = 'spam' >>> y = 99 >>> z = ['eggs'] >>> >>> print(x, y, z) spam 99 ['eggs']
print sep >>> print(x, y, z, sep='') spam99['eggs'] >>> >>> print(x, y, z, sep=', ') spam, 99, ['eggs']
str str
Print Operations | 299
www.it-ebooks.info
print end \n >>> print(x, y, z, end='') spam 99 ['eggs']>>> >>> >>> print(x, y, z, end=''); print(x, y, z) spam 99 ['eggs']spam 99 ['eggs'] >>> print(x, y, z, end='...\n') spam 99 ['eggs']... >>>
>>> print(x, y, z, sep='...', end='!\n') spam...99...['eggs']! >>> print(x, y, z, end='!\n', sep='...') spam...99...['eggs']!
file print >>> print(x, y, z, sep='...', file=open('data.txt', 'w')) >>> print(x, y, z) spam 99 ['eggs'] >>> print(open('data.txt').read()) spam...99...['eggs']
print >>> text = '%s: %-.4f, %05d' % ('Result', 3.14159, 42) >>> print(text) Result: 3.1416, 00042 >>> print('%s: %-.4f, %05d' % ('Result', 3.14159, 42)) Result: 3.1416, 00042
print print
The Python 2.6 print Statement
300 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
print print
Statement forms print print print >> sys.stdout
Python 2.6 statement
Python 3.0 equivalent
Interpretation
print x, y
print(x, y)
Print objects’ textual forms to sys.stdout; add a space between the items and an end-of-line at the end
print x, y,
print(x, y, end='')
Same, but don’t add end-of-line at end of text
print >> afile, x, y
print(x, y, file=afile)
Send text to myfile.write, not to sys.stdout.write
The 2.6 print statement in action print print C:\misc> c:\python26\python >>> >>> x = 'a' >>> y = 'b' >>> print x, y a b
print
>>> print x, y,; print x, y a b a b
Print Operations | 301
www.it-ebooks.info
>>> print x + y ab >>> print '%s...%s' % (x, y) a...b
print print
Print Stream Redirection
The Python “hello world” program
>>> print('hello world') hello world >>> print 'hello world' hello world
print >>> 'hello world' 'hello world'
print sys.stdout
>>> import sys >>> sys.stdout.write('hello world\n') hello world
302 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
write
sys.stdout print
Manual stream redirection sys.stdout print sys.stdout print(X, Y)
import sys sys.stdout.write(str(X) + ' ' + str(Y) + '\n')
str +
write
print sys.stdout print import sys sys.stdout = open('log.txt', 'a') ... print(x, y, x)
sys.stdout print print sys.stdout
sys.stdout write sys print
sys.stdout
print
stdout
sys.stdout write write
print print
Print Operations | 303
www.it-ebooks.info
sys.stdout print
Automatic stream redirection sys.stdout
sys.stdout C:\misc> c:\python30\python >>> import sys >>> temp = sys.stdout >>> sys.stdout = open('log.txt', 'a') >>> print('spam') >>> print(1, 2, 3) >>> sys.stdout.close() >>> sys.stdout = temp >>> print('back here') back here >>> print(open('log.txt').read()) spam 1 2 3
print file
print
write
sys.stdout print
print >>
log = open('log.txt', 'a') print(x, y, z, file=log) print(a, b, c) log = open('log.txt', 'a') print >> log, x, y, z print a, b, c
print
__stdout__
sys
sys.stdout sys.__stdout__
304 | Chapter 11: Assignments, Expressions, and Prints
sys.stdout sys
www.it-ebooks.info
write C:\misc> c:\python30\python >>> log = open('log.txt', 'w') >>> print(1, 2, 3, file=log) >>> print(4, 5, 6, file=log) >>> log.close() >>> print(7, 8, 9) 7 8 9 >>> print(open('log.txt').read()) 1 2 3 4 5 6
print sys.stderr
write
>>> import sys >>> sys.stderr.write(('Bad!' * 8) + '\n') Bad!Bad!Bad!Bad!Bad!Bad!Bad!Bad! >>> print('Bad!' * 8, file=sys.stderr) Bad!Bad!Bad!Bad!Bad!Bad!Bad!Bad!
write >>> >>> 1 2 >>> >>> 1 2 4 >>>
X = 1; Y = 2 print(X, Y)
>>> 4 >>> b'1 >>> b'1
open('temp2', 'w').write(str(X) + ' ' + str(Y) + '\n')
import sys sys.stdout.write(str(X) + ' ' + str(Y) + '\n') print(X, Y, file=open('temp1', 'w'))
print(open('temp1', 'rb').read()) 2\r\n' print(open('temp2', 'rb').read()) 2\r\n'
print print
Print Operations | 305
www.it-ebooks.info
Version-Neutral Printing print 2to3
print from __future__ import print_function
print
print
C:\misc> c:\python30\python >>> print('spam') spam >>> print('spam', 'ham', 'eggs') spam ham eggs
C:\misc> c:\python26\python >>> print('spam') spam >>> print('spam', 'ham', 'eggs') ('spam', 'ham', 'eggs')
>>> print('%s %s %s' % ('spam', 'ham', 'eggs')) spam ham eggs >>> print('{0} {1} {2}'.format('spam', 'ham', 'eggs')) spam ham eggs
306 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
print
print
Why You Will Care: print and stdout print sys.stdout write sys.stdout.write sys.stdout
sys.stdout print
write write
class FileFaker: def write(self, string): # Do something with printed text in string import sys sys.stdout = FileFaker() print(someObjects)
print sys.stdout write
file >>
print
sys.stdout
stdout
myobj = FileFaker() print(someObjects, file=myobj) myobj = FileFaker() print >> myobj, someObjects
input
sys.stdin read
input
while stdout
Print Operations | 307
www.it-ebooks.info
python script.py < inputfile > outputfile python script.py | filterProgram
Chapter Summary print
append
if
Test Your Knowledge: Quiz L = L.sort() print
Test Your Knowledge: Answers A = B = C = 0 A, B, C = 0, 0, 0 A = 0 B = 0 C = 0 A = 0; B = 0; C = 0
308 | Chapter 11: Assignments, Expressions, and Prints
www.it-ebooks.info
A = B = C = []
A.append(99)
sort
append None
L
L
None sorted
sys.stdout
print print >> file, X print
print(X, file=F)
Test Your Knowledge: Answers | 309
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 12
if Tests and Syntax Rules
if
if
if Statements if if if
General Format if
if if else
elif else if
else
if
if : elif : else:
311
www.it-ebooks.info
Basic Examples if if >>> if 1: ... print('true') ... true
...
1 else >>> if not 1: ... print('true') ... else: ... print('false') ... false
Multiway Branching if >>> >>> ... ... ... ... ... ... Run
x = 'killer rabbit' if x == 'roger': print("how's jessica?") elif x == 'bugs': print("what's up doc?") else: print('Run away! Run away!') away! Run away!
if
else
else else
elif if elif
switch
else
case if elif if
312 | Chapter 12: if Tests and Syntax Rules
www.it-ebooks.info
>>> choice = 'ham' >>> print({'spam': ... 'ham': ... 'eggs': ... 'bacon': 1.99
1.25, 1.99, 0.99, 1.10}[choice])
choice switch
if
>>> if choice == 'spam': ... print(1.25) ... elif choice == 'ham': ... print(1.99) ... elif choice == 'eggs': ... print(0.99) ... elif choice == 'bacon': ... print(1.10) ... else: ... print('Bad choice') ... 1.99
else
if in
get get
>>> branch = {'spam': 1.25, ... 'ham': 1.99, ... 'eggs': 0.99} >>> print(branch.get('spam', 'Bad choice')) 1.25 >>> print(branch.get('bacon', 'Bad choice')) Bad choice
in >>> >>> ... ... ... ... Bad
if choice = 'bacon' if choice in branch: print(branch[choice]) else: print('Bad choice') choice
if
if Statements | 313
www.it-ebooks.info
lambda
if
Python Syntax Rules if
if if
if
elif
else
#
#
314 | Chapter 12: if Tests and Syntax Rules
if
www.it-ebooks.info
Block Delimiters: Indentation Rules
x = 1 if x: y = 2 if y: print('block2') print('block1') print('block0')
Python Syntax Rules | 315
www.it-ebooks.info
if print
x = 'SPAM' if 'rubbery' in 'shrubbery': print(x * 8) x += 'NI' if x.endswith('NI'): x *= 2 print(x)
x = 'SPAM' if 'rubbery' in 'shrubbery': print(x * 8) x += 'NI' if x.endswith('NI'): x *= 2 print(x)
316 | Chapter 12: if Tests and Syntax Rules
if
www.it-ebooks.info
Avoid mixing tabs and spaces: New error checking in 3.0
-t -tt python –t main.py -tt
Statement Delimiters: Lines and Continuations
() {}
) }
[]
]
\
\
Python Syntax Rules | 317
www.it-ebooks.info
#
A Few Special Cases L = ["Good", "Bad", "Ugly"]
if a == b and c == d and d == e and f == g: print('olde')
\
if (a == b and c == d and d == e and e == f): print('new')
x
10 x
+4
318 | Chapter 12: if Tests and Syntax Rules
6
www.it-ebooks.info
x = 1 + 2 + 3 \ +4
x = 1; y = 2; print(x)
+
\ncccc'
S
S '\naaaa\nbbbb 'aaaabbbbcccc'
S = """ aaaa bbbb cccc""" S = ('aaaa' 'bbbb' 'cccc')
if if 1: print('hello')
Python Syntax Rules | 319
www.it-ebooks.info
Truth Tests if
None True and
False
1
0
or
X and Y X
Y
X or Y X
Y
not X X X
True
False
Y && ||
!
and True
False
True
False
or
>>> 2 < 3, 3 < 2 (True, False)
1 0 and
or if True
320 | Chapter 12: if Tests and Syntax Rules
False
www.it-ebooks.info
or
>>> (2, >>> 3 >>> {}
2 or 3, 3 or 2 3) [] or 3 [] or {}
2
3
and >>> (3, >>> [] >>> []
2 and 3, 3 and 2 2) [] and {} 3 and []
[] 3 [] if and
while
or
The if/else Ternary Expression if A
Y
Z
X
The if/else Ternary Expression | 321
www.it-ebooks.info
if X: A = Y else: A = Z
A = Y if X else Z
if Y X
Z
>>> >>> 't' >>> >>> 'f'
X
A = 't' if 'spam' else 'f' A A = 't' if '' else 'f' A
and
or
A = ((X and Y) or Z)
Y and or
Z
X
Y
Y Z
X
and or Y if X else Z if bool X
1
0
A = [Z, Y][bool(X)]
X if Y else Z
322 | Chapter 12: if Tests and Syntax Rules
Y ? X : Z
www.it-ebooks.info
>>> ['f', 't'][bool('')] 'f' >>> ['f', 't'][bool('spam')] 't'
Z
Y
X if else if
and or
Why You Will Care: Booleans or X = A or B or C or None
X
A B
C
None
or or X A
A
default
X = A or default
if f1() or f2(): ...
f1
f2 or
tmp1, tmp2 = f1(), f2() if tmp1 or tmp2: ...
((A and B) or C)
if
else
if X:
if X != '': True
False
1
0
The if/else Ternary Expression | 323
www.it-ebooks.info
X = False
while True:
__bool__ __len__
__bool__
__nonzero__
Chapter Summary if
if else while
for
Test Your Knowledge: Quiz if else True
False
Test Your Knowledge: Answers if
elif
def
lambda Y if X else Z
Z ((X and Y) or Z) Y
324 | Chapter 12: if Tests and Syntax Rules
if
Y and or
X
www.it-ebooks.info
() []
True
False
{}
1
0
Test Your Knowledge: Answers | 325
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 13
while and for Loops
while for
break range zip while
continue map for
for for filter
reduce
while Loops while
while
327
www.it-ebooks.info
General Format while else break
while : else:
Examples while print
while
True
>>> while True: ... print('Type Ctrl-C to stop me!')
while x != '': for >>> x = 'spam' >>> while x: ... print(x, end=' ') ... x = x[1:] ... spam pam am m
end=' ' a for
b range
>>> a=0; b=10 >>> while a < b: ... print(a, end=' ') ... a += 1 ... 0 1 2 3 4 5 6 7 8 9
break
328 | Chapter 13: while and for Loops
www.it-ebooks.info
while True: ...loop body... if exitTest(): break
break
break, continue, pass, and the Loop else break else
break pass
break continue pass Loop else block break
General Loop Format break
continue
while
while : if : break if : continue else:
break
continue
while
for
if
break, continue, pass, and the Loop else | 329
www.it-ebooks.info
pass pass
pass while True: pass
pass None while
if
pass pass try
class pass
def func1(): pass def func2(): pass
pass
... pass def func1(): ... def func2(): ... func1()
def func1(): ... def func2(): ... >>> X = ...
330 | Chapter 13: while and for Loops
www.it-ebooks.info
>>> X Ellipsis
... pass
None
continue continue continue % 8 6 4 2 0 x = 10 while x: x = x−1 if x % 2 != 0: continue print(x, end=' ')
continue if
print print
continue
continue continue print
if
x = 10 while x: x = x−1 if x % 2 == 0: print(x, end=' ')
break break break break input raw_input >>> while True: ... name = input('Enter name:') ... if name == 'stop': break ... age = input('Enter age: ') ... print('Hello', name, '=>', int(age) ** 2) ... Enter name:mel Enter age: 40
break, continue, pass, and the Loop else | 331
www.it-ebooks.info
Hello Enter Enter Hello Enter
mel => 1600 name:bob age: 30 bob => 900 name:stop
age
int input
input input
try
Loop else else
break y
x = y // 2 while x > 1: if y % x == 0: print(y, 'has factor', x) break x -= 1 else: print(y, 'is prime')
break else break else break
while
x
y
// /
332 | Chapter 13: while and for Loops
/
www.it-ebooks.info
More on the loop else else else
found = False while x and not found: if match(x[0]): print('Ni') found = True else: x = x[1:] if not found: print('not found')
else
else
while x: if match(x[0]): print('Ni') break x = x[1:] else: print('Not found')
else
if break
while while
else else x
if not x:
else else for
break, continue, pass, and the Loop else | 333
www.it-ebooks.info
Why You Will Care: Emulating C while Loops
while ((x = next()) != NULL) {...process x...}
=
== while break
while True: x = next() if not x: break ...process x...
x = True while x: x = next() if x: ...process x...
x = next() while x: ...process x... x = next()
for
for Loops for for
General Format for
334 | Chapter 13: while and for Loops
www.it-ebooks.info
for in : else:
for
for for
break for while
else break for
break continue while for
for in : if : break if : continue else:
Examples for
Basic usage for x print print
x
>>> for x in ["spam", "eggs", "ham"]: ... print(x, end=' ') ... spam eggs ham
+ *
for
for Loops | 335
www.it-ebooks.info
>>> >>> ... ... >>> 10 >>> >>> ... >>> 24
sum = 0 for x in [1, 2, 3, 4]: sum = sum + x sum prod = 1 for item in [1, 2, 3, 4]: prod *= item prod
Other data types for
for
>>> S = "lumberjack" >>> T = ("and", "I'm", "okay") >>> for x in S: print(x, end=' ') ... l u m b e r j a c k >>> for x in T: print(x, end=' ') ... and I'm okay
for
Tuple assignment in for loops
for >>> T = [(1, 2), (3, 4), (5, 6)] >>> for (a, b) in T: ... print(a, b) ... 1 2 3 4 5 6
(a,b) = (1,2) (a,b) = (3,4) zip
336 | Chapter 13: while and for Loops
www.it-ebooks.info
for items >>> D = {'a': 1, 'b': 2, 'c': 3} >>> for key in D: ... print(key, '=>', D[key]) ... a => 1 c => 3 b => 2 >>> list(D.items()) [('a', 1), ('c', 3), ('b', 2)] >>> for (key, value) in D.items(): ... print(key, '=>', value) ... a => 1 c => 3 b => 2
for for >>> T [(1, 2), (3, 4), (5, 6)] >>> for both in T: ... a, b = both ... print(a, b) ... 1 2 3 4 5 6
for >>> ((a, b), c) = ((1, 2), 3) >>> (1, 2, 3) >>> for ((a, b), c) in [((1, 2), 3), ((4, 5), 6)]: print(a, b, c) ... 1 2 3 4 5 6
for Loops | 337
www.it-ebooks.info
for
>>> for ((a, b), c) in [([1, 2], 3), ['XY', 6]]: print(a, b, c) ... 1 2 3 X Y 6
Python 3.0 extended sequence assignment in for loops for
for
>>> a, b, c = (1, 2, 3) >>> a, b, c (1, 2, 3) >>> for (a, b, c) in [(1, 2, 3), (4, 5, 6)]: ... print(a, b, c) ... 1 2 3 4 5 6
for >>> a, *b, c = (1, 2, 3, 4) >>> a, b, c (1, [2, 3], 4) >>> for (a, *b, c) in [(1, 2, 3, 4), (5, 6, 7, 8)]: ... print(a, b, c) ... 1 [2, 3] 4 5 [6, 7] 8
>>> for all in [(1, 2, 3, 4), (5, 6, 7, 8)]: ... a, b, c = all[0], all[1:3], all[3] ... print(a, b, c)
338 | Chapter 13: while and for Loops
www.it-ebooks.info
... 1 (2, 3) 4 5 (6, 7) 8
Nested for loops for else items
for
tests
>>> items = ["aaa", 111, (4, 5), 2.01] >>> tests = [(4, 5), 3.14] >>> >>> for key in tests: ... for item in items: ... if item == key: ... print(key, "was found") ... break ... else: ... print(key, "not found!") ... (4, 5) was found 3.14 not found!
if
break
else
else for if
for in
in >>> for key in tests: ... if key in items: ... print(key, "was found") ... else: ... print(key, "not found!") ... (4, 5) was found 3.14 not found!
for res
seq1
seq2
for Loops | 339
www.it-ebooks.info
>>> seq1 = "spam" >>> seq2 = "scam" >>> >>> res = [] >>> for x in seq1: ... if x in seq2: ... res.append(x) ... >>> res ['s', 'a', 'm']
seq1 seq2
Why You Will Care: File Scanners
read file = open('test.txt', 'r') print(file.read())
while for file = open('test.txt') while True: char = file.read(1) if not char: break print(char) for char in open('test.txt').read(): print(char)
for while file = open('test.txt') while True: line = file.readline() if not line: break print(line, end='') file = open('test.txt', 'rb') while True: chunk = file.read(10) if not chunk: break print(chunk)
340 | Chapter 13: while and for Loops
www.it-ebooks.info
for for line in open('test.txt').readlines(): print(line, end='') for line in open('test.txt'): print(line, end='')
readlines
open
file
file
xreadlines readlines xreadlines
Loop Coding Techniques for while
for while for range for zip for for
while for
Loop Coding Techniques | 341
www.it-ebooks.info
Counter Loops: while and range range for range list >>> list(range(5)), list(range(2, 5)), list(range(0, 10, 2)) ([0, 1, 2, 3, 4], [2, 3, 4], [0, 2, 4, 6, 8])
range
>>> list(range(−5, 5)) [−5, −4, −3, −2, −1, 0, 1, 2, 3, 4] >>> list(range(5, −5, −1)) [5, 4, 3, 2, 1, 0, −1, −2, −3, −4]
range for for
range range
list >>> for i in range(3): ... print(i, 'Pythons') ... 0 Pythons 1 Pythons 2 Pythons
range for >>> X = 'spam' >>> for item in X: print(item, end=' ') ... s p a m
for while >>> i = 0 >>> while i < len(X): ... print(X[i], end=' ') ... i += 1
342 | Chapter 13: while and for Loops
www.it-ebooks.info
... s p a m
for
range
>>> X 'spam' >>> len(X) 4 >>> list(range(len(X))) [0, 1, 2, 3] >>> >>> for i in range(len(X)): print(X[i], end=' ') ... s p a m
X X
X
Nonexhaustive Traversals: range and Slices for
for
while
range
for
>>> for item in X: print(item) ...
>>> S = 'abcdefghijk' >>> list(range(0, len(S), 2)) [0, 2, 4, 6, 8, 10] >>> for i in range(0, len(S), 2): print(S[i], end=' ') ... a c e g i k
S
range range
3
range for
S
Loop Coding Techniques | 343
www.it-ebooks.info
>>> S = 'abcdefghijk' >>> for c in S[::2]: print(c, end=' ') ... a c e g i k
range
Changing Lists: range range
for
for >>> L = [1, 2, 3, 4, 5] >>> ... ... >>> [1, >>> 6
for x in L: x += 1 L 2, 3, 4, 5] x
x
L
x x x
1
2
1
range len >>> L = [1, 2, 3, 4, 5] >>> for i in range(len(L)): ... L[i] += 1 ... >>> L [2, 3, 4, 5, 6]
for x in L: while >>> i = 0 >>> while i < len(L): ... L[i] += 1
344 | Chapter 13: while and for Loops
www.it-ebooks.info
... i += 1 ... >>> L [3, 4, 5, 6, 7]
range [x+1 for x in L]
L
Parallel Traversals: zip and map range
for zip
for zip
>>> L1 = [1,2,3,4] >>> L2 = [5,6,7,8]
zip range zip
list
>>> zip(L1, L2) >>> list(zip(L1, L2)) [(1, 5), (2, 6), (3, 7), (4, 8)]
for >>> ... ... 1 5 2 6 3 7 4 8
for (x, y) in zip(L1, L2): print(x, y, '--', x+y) -----
6 8 10 12
zip for zip (x, y) = (1, 5)
Loop Coding Techniques | 345
www.it-ebooks.info
L1
L2
while for zip zip
>>> T1, T2, T3 = (1,2,3), (4,5,6), (7,8,9) >>> T3 (7, 8, 9) >>> list(zip(T1, T2, T3)) [(1, 4, 7), (2, 5, 8), (3, 6, 9)]
zip
>>> S1 = 'abc' >>> S2 = 'xyz123' >>> >>> list(zip(S1, S2)) [('a', 'x'), ('b', 'y'), ('c', 'z')]
map equivalence in Python 2.6 map None >>> S1 = 'abc' >>> S2 = 'xyz123' >>> map(None, S1, S2) [('a', 'x'), ('b', 'y'), ('c', 'z'), (None, '1'), (None, '2'), (None,'3')]
map map map ord map
zip list
>>> list(map(ord, 'spam')) [115, 112, 97, 109]
346 | Chapter 13: while and for Loops
www.it-ebooks.info
>>> res = [] >>> for c in 'spam': res.append(ord(c)) >>> res [115, 112, 97, 109]
map None zip
map zip
Dictionary construction with zip zip zip
>>> D1 = {'spam':1, 'eggs':3, 'toast':5} >>> D1 {'toast': 5, 'eggs': 3, 'spam': 1} >>> >>> >>> >>>
D1 = {} D1['spam'] = 1 D1['eggs'] = 3 D1['toast'] = 5
>>> keys = ['spam', 'eggs', 'toast'] >>> vals = [1, 3, 5]
zip for >>> list(zip(keys, vals)) [('spam', 1), ('eggs', 3), ('toast', 5)] >>> D2 = {} >>> for (k, v) in zip(keys, vals): D2[k] = v ... >>> D2 {'toast': 5, 'eggs': 3, 'spam': 1}
Loop Coding Techniques | 347
www.it-ebooks.info
for dict >>> keys = ['spam', 'eggs', 'toast'] >>> vals = [1, 3, 5] >>> D3 = dict(zip(keys, vals)) >>> D3 {'toast': 5, 'eggs': 3, 'spam': 1}
dict
dict
Generating Both Offsets and Items: enumerate range for >>> S = 'spam' >>> offset = 0 >>> for item in S: ... print(item, ... offset += 1 ... s appears at offset p appears at offset a appears at offset m appears at offset
'appears at offset', offset) 0 1 2 3
enumerate >>> S = 'spam' >>> for (offset, item) in enumerate(S): ... print(item, 'appears at offset', offset) ... s appears at offset 0 p appears at offset 1 a appears at offset 2 m appears at offset 3
enumerate __next__
next
for
zip
(index, value)
348 | Chapter 13: while and for Loops
www.it-ebooks.info
>>> E = enumerate(S) >>> E >>> next(E) (0, 's') >>> next(E) (1, 'p') >>> next(E) (2, 'a')
>>> [c * i for (i, c) in enumerate(S)] ['', 'p', 'aa', 'mmm']
enumerate zip
Chapter Summary while
for
else
break
continue for
range zip map
enumerate
for range zip
Test Your Knowledge: Quiz while break
for
continue
else range
for
Test Your Knowledge: Quiz | 349
www.it-ebooks.info
Test Your Knowledge: Answers while
for while
for break while for else
continue while while
for
for break
break
else while for
range for
range range
for range
350 | Chapter 13: while and for Loops
www.it-ebooks.info
CHAPTER 14
Iterations and Comprehensions, Part 1
while
for
for for for
Iterators: A First Look for >>> for x in [1, 2, 3, 4]: print(x ** 2, end=' ') ... 1 4 9 16 >>> for x in (1, 2, 3, 4): print(x ** 3, end=' ') ... 1 8 27 64
351
www.it-ebooks.info
>>> for x in 'spam': print(x * 2, end=' ') ... ss pp aa mm
for for map
in
for
The Iteration Protocol: File Iterators readline readline >>> f = open('script1.py') >>> f.readline() 'import sys\n' >>> f.readline() 'print(sys.path)\n' >>> f.readline() 'x = 2\n' >>> f.readline() 'print(2 ** 33)\n' >>> f.readline() ''
__next__ __next__
StopIteration
>>> f = open('script1.py') >>> f.__next__() 'import sys\n' >>> f.__next__() 'print(sys.path)\n'
iter
352 | Chapter 14: Iterations and Comprehensions, Part 1
iter
next(I)
www.it-ebooks.info
>>> f.__next__() 'x = 2\n' >>> f.__next__() 'print(2 ** 33)\n' >>> f.__next__() Traceback (most recent call last): ...more exception text omitted... StopIteration
__next__
StopIteration for __next__
StopIteration for __next__
>>> for line in open('script1.py'): ... print(line.upper(), end='') ... IMPORT SYS PRINT(SYS.PATH) X = 2 PRINT(2 ** 33)
print
end=''
\n
for
readlines
>>> for line in open('script1.py').readlines(): ... print(line.upper(), end='') ... IMPORT SYS PRINT(SYS.PATH) X = 2 PRINT(2 ** 33)
readlines
Iterators: A First Look | 353
www.it-ebooks.info
while >>> f = open('script1.py') >>> while True: ... line = f.readline() ... if not line: break ... print(line.upper(), end='') ... ...same output...
for while
Manual Iteration: iter and next next X
next(X)
__next__ X.__next__()
>>> f = open('script1.py') >>> f.__next__() 'import sys\n' >>> f.__next__() 'print(sys.path)\n' >>> f = open('script1.py') >>> next(f) 'import sys\n' >>> next(f) 'print(sys.path)\n'
for iter iter for >>> >>> >>> 1 >>> 2
L = [1, 2, 3] I = iter(L) I.next() I.next()
354 | Chapter 14: Iterations and Comprehensions, Part 1
next
www.it-ebooks.info
>>> I.next() 3 >>> I.next() Traceback (most recent call last): ...more omitted... StopIteration
__next__ >>> f = open('script1.py') >>> iter(f) is f True >>> f.__next__() 'import sys\n'
iter >>> L = [1, 2, 3] >>> iter(L) is L False >>> L.__next__() AttributeError: 'list' object has no attribute '__next__' >>> I = iter(L) >>> I.__next__() 1 >>> next(I) 2
>>> L = [1, 2, 3] >>> >>> for X in L: ... print(X ** 2, end=' ') ... 1 4 9 >>> I = iter(L)
for
I.__next__
next(I)
os.popen for os.popen __getattr__
subprocess __next__
next os.popen2/3/4
subprocess.Popen
Iterators: A First Look | 355
www.it-ebooks.info
>>> while True: ... try: ... X = next(I) ... except StopIteration: ... break ... print(X ** 2, end=' ') ... 1 4 9
try for
X.next()
X.__next__()
X.next()
X.__next__()
X.next()
X.next() X.__next__() next(X)
Other Built-in Type Iterators
>>> D = {'a':1, 'b':2, 'c':3} >>> for key in D.keys(): ... print(key, D[key]) ... a 1 c 3 b 2
>>> I = iter(D) >>> next(I) 'a' >>> next(I) 'c' >>> next(I) 'b' >>> next(I) Traceback (most recent call last):
356 | Chapter 14: Iterations and Comprehensions, Part 1
next(X)
next(X)
www.it-ebooks.info
...more omitted... StopIteration
keys for >>> for key in D: ... print(key, D[key]) ... a 1 c 3 b 2
for os.popen >>> import os >>> P = os.popen('dir') >>> P.__next__() ' Volume in drive C is SQ004828V03\n' >>> P.__next__() ' Volume Serial Number is 08BE-3CD4\n' >>> next(P) TypeError: _wrap_close object is not an iterator
popen P.__next__()
P.next() next(P)
for
list >>> R = range(5) >>> R range(0, 5) >>> I = iter(R) >>> next(I) 0 >>> next(I) 1 >>> list(range(5)) [0, 1, 2, 3, 4]
Iterators: A First Look | 357
www.it-ebooks.info
enumerate >>> E = enumerate('spam') >>> E >>> I = iter(E) >>> next(I) (0, 's') >>> next(I) (1, 'p') >>> list(enumerate('spam')) [(0, 's'), (1, 'p'), (2, 'a'), (3, 'm')]
for
List Comprehensions: A First Look for range >>> L = [1, 2, 3, 4, 5] >>> for i in range(len(L)): ... L[i] += 10 ... >>> L [11, 12, 13, 14, 15]
>>> L = [x + 10 for x in L] >>> L [21, 22, 23, 24, 25]
for
358 | Chapter 14: Iterations and Comprehensions, Part 1
www.it-ebooks.info
List Comprehension Basics
for >>> L = [x + 10 for x in L]
x + 10 for for x in L L x x + 10
x
L
for >>> res = [] >>> for x in L: ... res.append(x + 10) ... >>> res [21, 22, 23, 24, 25]
for
Using List Comprehensions on Files readlines >>> f = open('script1.py') >>> lines = f.readlines()
List Comprehensions: A First Look | 359
www.it-ebooks.info
>>> lines ['import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(2 ** 33)\n']
\n
lines rstrip line[:−1] >>> lines = [line.rstrip() for line in lines] >>> lines ['import sys', 'print(sys.path)', 'x = 2', 'print(2 ** 33)']
for
next
rstrip rstrip
>>> lines = [line.rstrip() for line in open('script1.py')] >>> lines ['import sys', 'print(sys.path)', 'x = 2', 'print(2 ** 33)']
for
>>> [line.upper() for line in open('script1.py')] ['IMPORT SYS\n', 'PRINT(SYS.PATH)\n', 'X = 2\n', 'PRINT(2 ** 33)\n'] >>> [line.rstrip().upper() for line in open('script1.py')] ['IMPORT SYS', 'PRINT(SYS.PATH)', 'X = 2', 'PRINT(2 ** 33)'] >>> [line.split() for line in open('script1.py')] [['import', 'sys'], ['print(sys.path)'], ['x', '=', '2'], ['print(2', '**','33)']]
360 | Chapter 14: Iterations and Comprehensions, Part 1
www.it-ebooks.info
>>> [line.replace(' ', '!') for line in open('script1.py')] ['import!sys\n', 'print(sys.path)\n', 'x!=!2\n', 'print(2!**!33)\n'] >>> [('sys' in line, line[0]) for line in open('script1.py')] [(True, 'i'), (True, 'p'), (False, 'x'), (False, 'p')]
Extended List Comprehension Syntax for
if
if >>> lines = [line.rstrip() for line in open('script1.py') if line[0] == 'p'] >>> lines ['print(sys.path)', 'print(2 ** 33)']
if for for >>> res = [] >>> for line in open('script1.py'): ... if line[0] == 'p': ... res.append(line.rstrip()) ... >>> res ['print(sys.path)', 'print(2 ** 33)']
for
for for if x + y
x
y >>> [x + y for x in 'abc' for y in 'lmn'] ['al', 'am', 'an', 'bl', 'bm', 'bn', 'cl', 'cm', 'cn']
List Comprehensions: A First Look | 361
www.it-ebooks.info
>>> res = [] >>> for x in 'abc': ... for y in 'lmn': ... res.append(x + y) ... >>> res ['al', 'am', 'an', 'bl', 'bm', 'bn', 'cl', 'cm', 'cn']
for
Other Iteration Contexts
for
for >>> for line in open('script1.py'): ... print(line.upper(), end='') ... IMPORT SYS PRINT(SYS.PATH) X = 2 PRINT(2 ** 33)
in sorted
map zip
>>> uppers = [line.upper() for line in open('script1.py')] >>> uppers ['IMPORT SYS\n', 'PRINT(SYS.PATH)\n', 'X = 2\n', 'PRINT(2 ** 33)\n']
362 | Chapter 14: Iterations and Comprehensions, Part 1
www.it-ebooks.info
>>> map(str.upper, open('script1.py')) >>> list( map(str.upper, open('script1.py')) ) ['IMPORT SYS\n', 'PRINT(SYS.PATH)\n', 'X = 2\n', 'PRINT(2 ** 33)\n'] >>> 'y = 2\n' in open('script1.py') False >>> 'x = 2\n' in open('script1.py') True
map map
list map
for sorted
zip
enumerate filter
reduce zip enumerate
filter
map
>>> sorted(open('script1.py')) ['import sys\n', 'print(2 ** 33)\n', 'print(sys.path)\n', 'x = 2\n'] >>> list(zip(open('script1.py'), open('script1.py'))) [('import sys\n', 'import sys\n'), ('print(sys.path)\n', 'print(sys.path)\n'), ('x = 2\n', 'x = 2\n'), ('print(2 ** 33)\n', 'print(2 ** 33)\n')] >>> list(enumerate(open('script1.py'))) [(0, 'import sys\n'), (1, 'print(sys.path)\n'), (2, 'x = 2\n'), (3, 'print(2 ** 33)\n')] >>> list(filter(bool, open('script1.py'))) ['import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(2 ** 33)\n'] >>> import functools, operator >>> functools.reduce(operator.add, open('script1.py')) 'import sys\nprint(sys.path)\nx = 2\nprint(2 ** 33)\n'
zip filter
enumerate
reduce
sorted sorted sort
Other Iteration Contexts | 363
www.it-ebooks.info
map
sorted
sum any True
>>> sum([3, 2, 4, 1, 15 >>> any(['spam', '', True >>> all(['spam', '', False >>> max([3, 2, 5, 1, 5 >>> min([3, 2, 5, 1, 1
all max reduce
True min
5, 0]) 'ni']) 'ni']) 4]) 4])
max
min
>>> max(open('script1.py')) 'x = 2\n' >>> min(open('script1.py')) 'import sys\n'
list join
tuple
>>> list(open('script1.py')) ['import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(2 ** 33)\n'] >>> tuple(open('script1.py')) ('import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(2 ** 33)\n') >>> '&&'.join(open('script1.py')) 'import sys\n&&print(sys.path)\n&&x = 2\n&&print(2 ** 33)\n' >>> a, b, c, d = open('script1.py') >>> a, d
364 | Chapter 14: Iterations and Comprehensions, Part 1
www.it-ebooks.info
('import sys\n', 'print(2 ** 33)\n') >>> a, *b = open('script1.py') >>> a, b ('import sys\n', ['print(sys.path)\n', 'x = 2\n', 'print(2 ** 33)\n'])
dict
zip
set >>> set(open('script1.py')) {'print(sys.path)\n', 'x = 2\n', 'print(2 ** 33)\n', 'import sys\n'} >>> {line for line in open('script1.py')} {'print(sys.path)\n', 'x = 2\n', 'print(2 ** 33)\n', 'import sys\n'} >>> {ix: line for ix, line in enumerate(open('script1.py'))} {0: 'import sys\n', 1: 'print(sys.path)\n', 2: 'x = 2\n', 3: 'print(2 ** 33)\n'}
if >>> {line for line in open('script1.py') if line[0] == 'p'} {'print(sys.path)\n', 'print(2 ** 33)\n'} >>> {ix: line for (ix, line) in enumerate(open('script1.py')) if line[0] == 'p'} {1: 'print(sys.path)\n', 3: 'print(2 ** 33)\n'}
*arg
>>> def f(a, b, c, d): print(a, b, c, d, sep='&') ... >>> f(1, 2, 3, 4) 1&2&3&4 >>> f(*[1, 2, 3, 4]) 1&2&3&4 >>> f(*open('script1.py')) import sys &print(sys.path) &x = 2 &print(2 ** 33)
zip
zip
Other Iteration Contexts | 365
www.it-ebooks.info
zip >>> X = (1, 2) >>> Y = (3, 4) >>> >>> list(zip(X, Y)) [(1, 3), (2, 4)] >>> >>> A, B = zip(*zip(X, Y)) >>> A (1, 2) >>> B (3, 4)
range
New Iterables in Python 3.0 keys values items range map zip filter
list(...) >>> zip('abc', 'xyz') >>> list(zip('abc', 'xyz')) [('a', 'x'), ('b', 'y'), ('c', 'z')]
zip
366 | Chapter 14: Iterations and Comprehensions, Part 1
www.it-ebooks.info
The range Iterator range xrange list(range(...)) C:\\misc> c:\python30\python >>> R = range(10) >>> R range(0, 10) >>> >>> 0 >>> 1 >>> 2
I = iter(R) next(I) next(I) next(I)
>>> list(range(10)) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
range len list(...) >>> len(R) 10 >>> R[0] 0 >>> R[-1] 9 >>> next(I) 3 >>> I.__next__() 4
xrange range range
xrange
range file.xread lines()
New Iterables in Python 3.0 | 367
www.it-ebooks.info
The map, zip, and filter Iterators range
map zip
filter range
map list(...) >>> M = map(abs, (-1, 0, 1)) >>> M >>> next(M) 1 >>> next(M) 0 >>> next(M) 1 >>> next(M) StopIteration >>> for x in M: print(x) ... >>> >>> ... 1 0 1 >>> [1,
M = map(abs, (-1, 0, 1)) for x in M: print(x)
list(map(abs, (-1, 0, 1))) 0, 1]
zip >>> Z = zip((1, 2, 3), (10, 20, 30)) >>> Z >>> list(Z) [(1, 10), (2, 20), (3, 30)] >>> for pair in Z: print(pair) ... >>> Z = zip((1, 2, 3), (10, 20, 30)) >>> for pair in Z: print(pair) ... (1, 10)
368 | Chapter 14: Iterations and Comprehensions, Part 1
www.it-ebooks.info
(2, 20) (3, 30) >>> >>> (1, >>> (2,
Z = zip((1, 2, 3), (10, 20, 30)) next(Z) 10) next(Z) 20)
filter True True >>> filter(bool, ['spam', '', 'ni']) >>> list(filter(bool, ['spam', '', 'ni'])) ['spam', 'ni']
filter
Multiple Versus Single Iterators range len iter >>> R = range(3) >>> next(R) TypeError: range object is not an iterator >>> >>> 0 >>> 1 >>> >>> 0 >>> 2
I1 = iter(R) next(I1) next(I1) I2 = iter(R) next(I2) next(I1)
zip map >>> >>> >>> >>> (1, >>> (2, >>>
filter
Z = zip((1, 2, 3), (10, 11, 12)) I1 = iter(Z) I2 = iter(Z) next(I1) 10) next(I1) 11) next(I2)
New Iterables in Python 3.0 | 369
www.it-ebooks.info
(3, 12) >>> M = map(abs, (-1, 0, 1)) >>> I1 = iter(M); I2 = iter(M) >>> print(next(I1), next(I1), next(I1)) 1 0 1 >>> next(I2) StopIteration >>> R = range(3) >>> I1, I2 = iter(R), iter(R) >>> [next(I1), next(I1), next(I1)] [0 1 2] >>> next(I2) 0
iter map
zip
range
Dictionary View Iterators keys values
>>> D = dict(a=1, b=2, c=3) >>> D {'a': 1, 'c': 3, 'b': 2} >>> K = D.keys() >>> K >>> next(K) TypeError: dict_keys object is not an iterator >>> I = iter(K) >>> next(I) 'a' >>> next(I) 'c' >>> for k in D.keys(): print(k, end=' ') ... a c b
370 | Chapter 14: Iterations and Comprehensions, Part 1
items
www.it-ebooks.info
list >>> K = D.keys() >>> list(K) ['a', 'c', 'b'] >>> V = D.values() >>> V >>> list(V) [1, 3, 2] >>> list(D.items()) [('a', 1), ('c', 3), ('b', 2)] >>> for (k, v) in D.items(): print(k, v, end=' ') ... a 1 c 3 b 2
keys >>> D {'a': 1, 'c': 3, 'b': 2} >>> I = iter(D) >>> next(I) 'a' >>> next(I) 'c' >>> for key in D: print(key, end=' ') ... a c b
keys list
sorted
>>> D {'a': 1, 'c': 3, 'b': 2} >>> for k in sorted(D.keys())): print(k, D[k], end=' ') ... a 1 b 2 c 3 >>> D {'a': 1, 'c': 3, 'b': 2} >>> for k in sorted(D): print(k, D[k], end=' ') ... a 1 b 2 c 3
New Iterables in Python 3.0 | 371
www.it-ebooks.info
Other Iterator Topics
yield
__iter__
Chapter Summary for
Test Your Knowledge: Quiz for for
372 | Chapter 14: Iterations and Comprehensions, Part 1
__getitem__
www.it-ebooks.info
Test Your Knowledge: Answers for __next__ StopIteration
next for
for for for for sorted sum any join __next__
in all
map list
tuple
for next
Test Your Knowledge: Answers | 373
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 15
The Documentation Interlude
Python Documentation Sources
375
www.it-ebooks.info
Form
Role
# comments
In-file documentation
The dir function
Lists of attributes available in objects
Docstrings: __doc__
In-file documentation attached to objects
PyDoc: The help function
Interactive help for objects
PyDoc: HTML reports
Module documentation in a browser
The standard manual set
Official language and library descriptions
Web resources
Online tutorials, examples, and so on
Published books
Commercially available reference texts
# Comments #
#
The dir Function dir sys dir >>> import sys >>> dir(sys) ['__displayhook__', '__doc__', '__excepthook__', '__name__', '__package__', '__stderr__', '__stdin__', '__stdout__', '_clear_type_cache', '_current_frames', '_getframe', 'api_version', 'argv', 'builtin_module_names', 'byteorder', 'call_tracing', 'callstats', 'copyright', 'displayhook', 'dllhandle', 'dont_write_bytecode', 'exc_info', 'excepthook', 'exec_prefix', 'executable', 'exit', 'flags', 'float_info', 'getcheckinterval', 'getdefaultencoding', ...more names omitted...]
376 | Chapter 15: The Documentation Interlude
www.it-ebooks.info
dir
>>> dir([]) ['__add__', '__class__', '__contains__', ...more... 'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort'] >>> dir('') ['__add__', '__class__', '__contains__', ...more... 'capitalize', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', ' maketrans', 'partition', 'replace', 'rfind', 'rindex', 'rjust', ...more names omitted...]
dir
dir >>> dir(str) == dir('') True >>> dir(list) == dir([]) True
str
list
dir
Docstrings: __doc__ #
# __doc__
Python Documentation Sources | 377
www.it-ebooks.info
User-defined docstrings
def """ Module documentation Words Go Here """ spam = 40 def square(x): """ function documentation can we have your liver then? """ return x ** 2 # square class Employee: "class documentation" pass print(square(4)) print(square.__doc__)
__doc__ __doc__ >>> import docstrings 16 function documentation can we have your liver then? >>> print(docstrings.__doc__) Module documentation Words Go Here >>> print(docstrings.square.__doc__) function documentation can we have your liver then? >>> print(docstrings.Employee.__doc__) class documentation
378 | Chapter 15: The Documentation Interlude
class
www.it-ebooks.info
print
def
class module.class.method.__doc__
Docstring standards
Built-in docstrings dir __doc__ >>> import sys >>> print(sys.__doc__) This module provides access to some objects used or maintained by the interpreter and to functions that interact strongly with the interpreter. Dynamic objects: argv -path -modules ...more
command line arguments; argv[0] is the script pathname if known module search path; path[0] is the script directory, else '' -- dictionary of loaded modules text omitted...
__doc__ >>> print(sys.getrefcount.__doc__) getrefcount(object) -> integer Return the reference count of object. The count returned is generally one higher than you might expect, because it includes the (temporary) ...more text omitted...
Python Documentation Sources | 379
www.it-ebooks.info
>>> print(int.__doc__) int(x[, base]) -> integer Convert a string or number to an integer, if possible. A floating point argument will be truncated towards zero (this does not include a ...more text omitted... >>> print(map.__doc__) map(func, *iterables) --> map object Make an iterator that computes the function using arguments from each of the iterables. Stops when the shortest iterable is exhausted.
help
PyDoc: The help Function
help help >>> import sys >>> help(sys.getrefcount) Help on built-in function getrefcount in module sys: getrefcount(...) getrefcount(object) -> integer Return the reference count of object. The count returned is generally one higher than you might expect, because it includes the (temporary) ...more omitted...
sys sys
sys help
380 | Chapter 15: The Documentation Interlude
help
www.it-ebooks.info
>>> help(sys) Help on built-in module sys: NAME
sys
FILE
(built-in)
MODULE DOCS http://docs.python.org/library/sys DESCRIPTION This module provides access to some objects used or maintained by the interpreter and to functions that interact strongly with the interpreter. ...more omitted... FUNCTIONS __displayhook__ = displayhook(...) displayhook(object) -> None Print an object to sys.stdout and also save it in builtins. ...more omitted... DATA
__stderr__ = __stdin__ = __stdout__ = ...more omitted...
help dict str
list >>> help(dict) Help on class dict in module builtins: class dict(object) | dict() -> new empty dictionary. | dict(mapping) -> new dictionary initialized from a mapping object's ...more omitted... >>> help(str.replace) Help on method_descriptor: replace(...) S.replace (old, new[, count]) -> str Return a copy of S with all occurrences of substring ...more omitted... >>> help(ord)
Python Documentation Sources | 381
www.it-ebooks.info
Help on built-in function ord in module builtins: ord(...) ord(c) -> integer Return the integer ordinal of a one-character string.
help
>>> import docstrings >>> help(docstrings.square) Help on function square in module docstrings: square(x) function documentation can we have your liver then? >>> help(docstrings.Employee) Help on class Employee in module docstrings: class Employee(builtins.object) | class documentation | | Data descriptors defined here: ...more omitted... >>> help(docstrings) Help on module docstrings: NAME
docstrings
FILE
c:\misc\docstrings.py
DESCRIPTION Module documentation Words Go Here CLASSES builtins.object Employee class Employee(builtins.object) | class documentation | | Data descriptors defined here: ...more omitted... FUNCTIONS square(x) function documentation
382 | Chapter 15: The Documentation Interlude
www.it-ebooks.info
can we have your liver then? DATA
spam = 40
PyDoc: HTML Reports help
pydoc.py
-g
sys.path
glob
Python Documentation Sources | 383
www.it-ebooks.info
help
384 | Chapter 15: The Documentation Interlude
www.it-ebooks.info
Python Documentation Sources | 385
www.it-ebooks.info
The Standard Manual Set
386 | Chapter 15: The Documentation Interlude
www.it-ebooks.info
Web Resources
Published Books
Common Coding Gotchas
: if while for
Common Coding Gotchas | 387
www.it-ebooks.info
...
if
while
if (X==1):
{} for for x in seq: while range for
for
while
while
range
while a = b = [] a += [1, 2]
list.append list.sort None append
mylist = mylist.append(X) mylist None
for k in D.keys().sort(): sort
keys sort None
388 | Chapter 15: The Documentation Interlude
None
www.it-ebooks.info
sorted Ks = list(D.keys()) Ks.sort()
for k in Ks: keys
function() function
file.close
import
file.close()
import mod
import mod.py
import
Chapter Summary help
Test Your Knowledge: Quiz
Test Your Knowledge: Quiz | 389
www.it-ebooks.info
Test Your Knowledge: Answers
__doc__ help
dir(X)
Test Your Knowledge: Part III Exercises
for S
390 | Chapter 15: The Documentation Interlude
ord(character)
www.it-ebooks.info
map(ord, S)
for i in range(50): print('hello %d\n\a' % i)
for keys
sort
sorted while
found L = [1, 2, 4, 8, 16, 32, 64] X = 5 found = False i = 0 while not found and i < len(L): if 2 ** X == L[i]: found = True else: i = i+1 if found: print('at index', i) else: print(X, 'not found') C:\book\tests> python power.py at index 5
while
else
found
if for index
L.index(X)
else X
L
Test Your Knowledge: Part III Exercises | 391
www.it-ebooks.info
in 2 in [1,2,3] for
append
L 2 ** X map(function, list) map(lambda x: 2 ** x, range(7)) lambda
392 | Chapter 15: The Documentation Interlude
www.it-ebooks.info
PART IV
Functions
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 16
Function Basics
Statement
Examples
Calls
myfunc('spam', 'eggs', meat=ham)
def, return
def adder(a, b=1, *c): return a + b + c[0]
global
def changer(): global x; x = 'new'
nonlocal
def changer(): nonlocal x; x = 'new'
yield
def squares(x): for i in range(x): yield i ** 2
lambda
funcs = [lambda x: x**2, lambda x: x*3]
395
www.it-ebooks.info
Why Use Functions?
Coding Functions open len
396 | Chapter 16: Function Basics
www.it-ebooks.info
def def
def def def
while
def
if def
def def
lambda lambda def return return yield yield
global
global nonlocal nonlocal def
Coding Functions | 397
www.it-ebooks.info
def Statements def def (arg1, arg2,... argN):
def
def
return def (arg1, arg2,... argN): ... return
return return return
398 | Chapter 16: Function Basics
www.it-ebooks.info
return
None
yield
def Executes at Runtime def def def def
if
if test: def func(): ... else: def func(): ... ... func()
def
def
=
def
othername = func othername()
def func(): ... func() func.attr = value
Coding Functions | 399
www.it-ebooks.info
A First Example: Definitions and Calls def
Definition times >>> def times(x, y): ... return x * y ...
def times
Calls def
>>> times(2, 4) 8
times x 2 y
4 return
2 * 4
8
>>> x = times(3.14, 4) >>> x 12.56
>>> times('Ni', 4) 'NiNiNiNi'
400 | Chapter 16: Function Basics
www.it-ebooks.info
x
y
* times times
Polymorphism in Python x
x * y y
times
*
*
times
*
*
A First Example: Definitions and Calls | 401
www.it-ebooks.info
type
A Second Example: Intersecting Sequences for
Definition for
def intersect(seq1, seq2): res = [] for x in seq1: if x in seq2: res.append(x) return res
def
402 | Chapter 16: Function Basics
www.it-ebooks.info
return
Calls def def >>> s1 = "SPAM" >>> s2 = "SCAM" >>> intersect(s1, s2) ['S', 'A', 'M']
>>> [x for x in s1 if x in s2] ['S', 'A', 'M']
Polymorphism Revisited intersect >>> x = intersect([1, 2, 3], (1, 4)) >>> x [1]
intersect
intersect
for in
A Second Example: Intersecting Sequences | 403
www.it-ebooks.info
Local Variables res
intersect def intersect
res seq1 for
seq2 x
return res
intersect
Chapter Summary def
return def
return
file.readlines() in open
file.seek(0) in __iter__
404 | Chapter 16: Function Basics
__contains__ __getitem__
www.it-ebooks.info
Test Your Knowledge: Quiz return
Test Your Knowledge: Answers
def
def def if None return None
Test Your Knowledge: Answers | 405
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 17
Scopes
Python Scope Basics
def
def
407
www.it-ebooks.info
def
def def
X def X
def
def def def
X = 99 X X
X = 88 def
X = 99 def func(): X = 88
X
Scope Rules def open
408 | Chapter 17: Scopes
www.it-ebooks.info
def lambda
global def nonlocal
def __builtin__
__main__
=
import
def def
L L = X
L L
L.append(X) L
L global
nonlocal
Python Scope Basics | 409
www.it-ebooks.info
Name Resolution: The LEGB Rule def
global
nonlocal def
lambda
def lambda
def
lambda nonlocal spam object.spam
def
nonlocal
410 | Chapter 17: Scopes
www.it-ebooks.info
.
Scope Example # Global scope X = 99 def func(Y): # Local scope Z = X + Y return Z func(1)
X func X func def
func
Python Scope Basics | 411
www.it-ebooks.info
Y Z Y
Z Z
=
Y
Y
Z
The Built-in Scope builtins builtins
builtins
builtins dir >>> import builtins >>> dir(builtins) ['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BufferError', 'BytesWarning', 'DeprecationWarning', 'EOFError', 'Ellipsis', ...many more names omitted... 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']
None True
False
builtins >>> zip
412 | Chapter 17: Scopes
www.it-ebooks.info
>>> import builtins >>> builtins.zip
open def hider(): open = 'spam' ... open('data.txt')
open
X = 88 def func(): X = 99 func() print(X)
X X global
nonlocal
def
builtins
__builtin__ __builtins__
builtins
__builtin__
builtins __builtins__ is builtins __builtins__ is __builtin__ True
True
dir(__builtins__) builtins
Python Scope Basics | 413
www.it-ebooks.info
Breaking the Universe in Python 2.6 True
False True = False True __builtin__.True = False
True
False True
False
None
open
The global Statement global
nonlocal global
global
global
def nonlocal def
global
X = 88 def func(): global X X = 99
414 | Chapter 17: Scopes
global
www.it-ebooks.info
func() print(X)
global X
X
def
def global
y, z = 1, 2 def all_global(): global x x = y + z
x y
z
all_global y x global
y
z global x
z x x
Minimize Global Variables global def
X = 99 def func1(): global X X = 88 def func2(): global X X = 77
X X
The global Statement | 415
www.it-ebooks.info
Minimize Cross-File Changes
X = 99 import first print(first.X) first.X = 88
_thread threading
416 | Chapter 17: Scopes
queue thread threading
Queue
www.it-ebooks.info
X
X
X
X = 99 def setX(new): global X X = new import first first.setX(88)
X
The global Statement | 417
www.it-ebooks.info
Other Ways to Access Globals global sys.modules
var = 99 def local(): var = 0 def glob1(): global var var += 1 def glob2(): var = 0 import thismod thismod.var += 1 def glob3(): var = 0 import sys glob = sys.modules['thismod'] glob.var += 1 def test(): print(var) local(); glob1(); glob2(); glob3() print(var)
>>> import thismod >>> thismod.test() 99 102 >>> thismod.var 102
global global nonlocal
418 | Chapter 17: Scopes
www.it-ebooks.info
Scopes and Nested Functions def
Nested Scope Details X
X
builtins global X = value
X
X X X
X global nonlocal
Nested Scope Examples X = 99 def f1(): X = 88 def f2(): print(X) f2() f1()
def def
Scopes and Nested Functions | 419
www.it-ebooks.info
def
f1 f2
f1
f2
f1 f2
X
X
f1 def
X
f2
X
f1
def f1(): X = 88 def f2(): print(X) return f2 action = f1() action()
action f2
f2 X
Factory functions
>>> def maker(N): ... def action(X): ... return X ** N ... return action ...
>>> f = maker(2) >>> f
420 | Chapter 17: Scopes
f1
f1
f1
www.it-ebooks.info
def >>> f(3) 9 >>> f(4) 16
action
maker 2
maker
N
maker
action action
N
>>> g = maker(3) >>> g(3) 27 >>> f(3) 9
g
3
2
f N
maker
lambda def
Retaining enclosing scopes’ state with defaults def
f2 f2
f1
Scopes and Nested Functions | 421
www.it-ebooks.info
def f1(): x = 88 def f2(x=x): print(x) f2() f1()
arg = val arg
def
val f2
arg
x=x
x
x
x def f1
x
f1
x
88
def def def
def
def
>>> def f1(): ... x = 88 ... f2(x) ... >>> def f2(x): ... print(x) ... >>> f1() 88
def
lambda
def
Nested scopes and lambdas def lambda lambda def
422 | Chapter 17: Scopes
www.it-ebooks.info
def def
lambda lambda
def func(): x = 4 action = (lambda n: x ** n) return action x = func() print(x(2))
lambda
def
def func(): x = 4 action = (lambda n, x=x: x ** n) return action
lambda def lambda
Scopes versus defaults with loop variables lambda
def
i >>> def makeActions(): ... acts = [] ... for i in range(5): ... acts.append(lambda x: i ** x) ... return acts ... >>> acts = makeActions() >>> acts[0]
Scopes and Nested Functions | 423
www.it-ebooks.info
i >>> acts[0](2) 16 >>> acts[2](2) 16 >>> acts[4](2) 16
i >>> ... ... ... ... ... >>> >>> 0 >>> 4 >>> 16
def makeActions(): acts = [] for i in range(5): acts.append(lambda x, i=i: i ** x) return acts acts = makeActions() acts[0](2) acts[2](2) acts[4](2)
lambda
Arbitrary scope nesting def >>> def f1(): ... x = 99 ... def f2(): ... def f3(): ... print(x) ... f3()
def f(a=[])
424 | Chapter 17: Scopes
www.it-ebooks.info
... f2() ... >>> f1() 99
def
The nonlocal Statement nonlocal
def
nonlocal nonlocal nonlocal
global
def
global global
global nonlocal def
nonlocal def
nonlocal Basics nonlocal def func(): nonlocal name1, name2, ...
def def nonlocal
The nonlocal Statement | 425
www.it-ebooks.info
nonlocal def
nonlocal nonlocal def
global nonlocal nonlocal def global
nonlocal
global nonlocal
def nonlocal def
def def
nonlocal nonlocal global
nonlocal
global
nonlocal
def
def
nonlocal in Action def state
426 | Chapter 17: Scopes
tester nested
nested tester
www.it-ebooks.info
C:\\misc>c:\python30\python >>> def tester(start): ... state = start ... def nested(label): ... print(label, state) ... return nested ... >>> F = tester(0) >>> F('spam') spam 0 >>> F('ham') ham 0
def >>> def tester(start): ... state = start ... def nested(label): ... print(label, state) ... state += 1 ... return nested ... >>> F = tester(0) >>> F('spam') UnboundLocalError: local variable 'state' referenced before assignment
Using nonlocal for changes state
tester
nonlocal
nested tester
nested F >>> def tester(start): ... state = start ... def nested(label): ... nonlocal state ... print(label, state) ... state += 1 ... return nested ... >>> F = tester(0) >>> F('spam') spam 0 >>> F('ham') ham 1 >>> F('eggs') eggs 2
tester state nested
The nonlocal Statement | 427
www.it-ebooks.info
state >>> G = tester(42) >>> G('spam') spam 42 >>> G('eggs') eggs 43 >>> F('bacon') bacon 3
Boundary cases global
nonlocal def
nonlocal >>> def tester(start): ... def nested(label): ... nonlocal state ... state = 0 ... print(label, state) ... return nested ... SyntaxError: no binding for nonlocal 'state' found >>> ... ... ... ... ... ... >>> >>> abc >>> 0
def tester(start): def nested(label): global state state = 0 print(label, state) return nested F = tester(0) F('abc') 0 state
nonlocal def >>> spam = 99 >>> def tester(): ... def nested(): ... nonlocal spam ... print('Current=', spam) ... spam += 1 ... return nested ... SyntaxError: no binding for nonlocal 'spam' found
428 | Chapter 17: Scopes
def
www.it-ebooks.info
spam
tester
Why nonlocal?
nonlocal
nonlocal
tester
def tester(start): state = start def nested(label): nonlocal state print(label, state) state += 1 return nested F = tester(0) F('spam')
Shared state with globals nonlocal >>> def tester(start): ... global state ... state = start ... def nested(label): ... global state ... print(label, state) ... state += 1 ... return nested ... >>> F = tester(0) >>> F('spam')
The nonlocal Statement | 429
www.it-ebooks.info
spam 0 >>> F('eggs') eggs 1
global
tester state
state
>>> G = tester(42) >>> G('toast') toast 42 >>> G('bacon') bacon 43 >>> F('ham') ham 44
nonlocal state
global
tester
State with classes (preview)
tester nested def class self >>> class tester: ... def __init__(self, start): ... self.state = start ... def nested(self, label): ... print(label, self.state) ... self.state += 1 ... >>> F = tester(0) >>> F.nested('spam') spam 0 >>> F.nested('ham') ham 1 >>> G = tester(42) >>> G.nested('toast') toast 42
430 | Chapter 17: Scopes
def
class
www.it-ebooks.info
>>> G.nested('bacon') bacon 43 >>> F.nested('eggs') eggs 2 >>> F.state 3
__call__ >>> class tester: ... def __init__(self, start): ... self.state = start ... def __call__(self, label): ... print(label, self.state) ... self.state += 1 ... >>> H = tester(99) >>> H('juice') juice 99 >>> H('pancakes') pancakes 100
__call__
def def
State with function attributes
def >>> def tester(start): ... def nested(label): ... print(label, nested.state) ... nested.state += 1
The nonlocal Statement | 431
www.it-ebooks.info
... nested.state = start ... return nested ... >>> F = tester(0) >>> F('spam') spam 0 >>> F('ham') ham 1 >>> F.state 2 >>> >>> G = tester(42) >>> G('eggs') eggs 42 >>> F('ham') ham 2
nested tester
nested nested.state nested nonlocal
Chapter Summary
432 | Chapter 17: Scopes
nested nested
www.it-ebooks.info
Test Your Knowledge: Quiz >>> X = 'Spam' >>> def func(): ... print(X) ... >>> func()
>>> >>> ... ... >>> >>>
X = 'Spam' def func(): X = 'NI!'
>>> >>> ... ... ... >>> >>>
X = 'Spam' def func(): X = 'NI' print(X)
>>> >>> ... ... ... >>> >>>
X = 'Spam' def func(): global X X = 'NI'
>>> >>> ... ... ... ... ... >>> >>>
X = 'Spam' def func(): X = 'NI' def nested(): print(X) nested()
func() print(X)
func() print(X)
func() print(X)
func() X
Test Your Knowledge: Quiz | 433
www.it-ebooks.info
>>> def func(): ... X = 'NI' ... def nested(): ... nonlocal X ... X = 'Spam' ... nested() ... print(X) ... >>> func()
Test Your Knowledge: Answers 'Spam' 'Spam' print 'NI'
'Spam' print 'NI' 'NI'
'Spam'
print print 'Spam' X X 'NI'
434 | Chapter 17: Scopes
nonlocal X
www.it-ebooks.info
CHAPTER 18
Arguments
Argument-Passing Basics
435
www.it-ebooks.info
Arguments and Shared References >>> ... ... >>> >>> >>> 88
def f(a): a = 99 b = 88 f(b) print(b)
a f(b)
88
a
a
a a=99
436 | Chapter 18: Arguments
b
www.it-ebooks.info
>>> ... ... ... >>> >>> >>> >>> (1,
def changer(a, b): a = 2 b[0] = 'spam' X = 1 L = [1, 2] changer(X, L) X, L ['spam', 2])
changer
a b
a a X b L b[0] L changer
b
b L L
>>> >>> >>> >>> 1
X = 1 a = X a = 2 print(X)
>>> L = [1, 2] >>> b = L >>> b[0] = 'spam' >>> print(L) ['spam', 2]
Argument-Passing Basics | 437
www.it-ebooks.info
Avoiding Mutable Argument Changes
L = [1, 2] changer(X, L[:])
def changer(a, b): b = b[:]
438 | Chapter 18: Arguments
www.it-ebooks.info
a = 2 b[0] = 'spam'
L = [1, 2] changer(X, tuple(L))
tuple
Simulating Output Parameters return return
>>> ... ... ... ... >>> >>> >>> >>> (2,
def multiple(x, y): x = 2 y = [3, 4] return x, y X = 1 L = [1, 2] X, L = multiple(X, L) X, L [3, 4])
Argument-Passing Basics | 439
www.it-ebooks.info
X
def f((a, (b, c))):
f((1, (2, 3)))
a b
c
1 2
3 f(T)
def def f(T): (a, (b, c)) = T
def f((a, [b, c])): lambda
for
Special Argument-Matching Modes def
440 | Chapter 18: Arguments
L
www.it-ebooks.info
The Basics
name=value
name=value *
* *
Special Argument-Matching Modes | 441
www.it-ebooks.info
Matching Syntax
Syntax
Location
Interpretation
func(value)
Caller
Normal argument: matched by position
func(name=value)
Caller
Keyword argument: matched by name
func(*sequence)
Caller
Pass all objects in sequence as individual positional arguments
func(**dict)
Caller
Pass all key/value pairs in dict as individual keyword arguments
def func(name)
Function
Normal argument: matches any passed value by position or name
def func(name=value)
Function
Default argument value, if not passed in the call
def func(*name)
Function
Matches and collects remaining positional arguments in a tuple
def func(**name)
Function
Matches and collects remaining keyword arguments in a dictionary
def func(*args, name)
Function
Arguments that must be passed by keyword only in calls (3.0)
def func(*, name=value)
name=value *sequence
name name=value *name **name *name
*
print
442 | Chapter 18: Arguments
**dict
www.it-ebooks.info
* *
The Gritty Details
value *sequence
name=value **dict
name *
name=value name name=value
*name
**name **arg
*name **name
Special Argument-Matching Modes | 443
www.it-ebooks.info
name:value
name:value=default
def f()->value
Keyword and Default Examples
>>> def f(a, b, c): print(a, b, c) ...
a
1 b
2
print >>> f(1, 2, 3) 1 2 3
Keywords
>>> f(c=3, b=2, a=1) 1 2 3
c=3
3 c 3
444 | Chapter 18: Arguments
c c
www.it-ebooks.info
>>> f(1, c=3, b=2) 1 2 3
a b
c
func(name='Bob', age=40, job='dev')
Defaults
>>> def f(a, b=2, c=3): print(a, b, c) ...
a b b
c >>> 1 2 >>> 1 2
2
c
3
f(1) 3 f(a=1) 3
c >>> 1 4 >>> 1 4
f(1, 4) 3 f(1, 4, 5) 5
>>> f(1, c=6) 1 2 6
a
1
c
6
b
2
name=value
Special Argument-Matching Modes | 445
www.it-ebooks.info
Combining keywords and defaults spam toast
eggs ham def func(spam, eggs, toast=0, ham=0): print((spam, eggs, toast, ham)) func(1, 2) func(1, ham=1, eggs=0) func(spam=1, eggs=0) func(toast=1, eggs=2, spam=3) func(1, 2, 3, 4)
spam
eggs name=value
def
Arbitrary Arguments Examples *
**
Collecting arguments
>>> def f(*args): print(args) ...
args for >>> f() () >>> f(1) (1,) >>> f(1, 2, 3, 4) (1, 2, 3, 4)
** ** keys
446 | Chapter 18: Arguments
www.it-ebooks.info
>>> def f(**args): print(args) ... >>> f() {} >>> f(a=1, b=2) {'a': 1, 'b': 2}
* 2
3
pargs
** 1 x
a y
kargs >>> def f(a, *pargs, **kargs): print(a, pargs, kargs) ... >>> f(1, 2, 3, x=1, y=2) 1 (2, 3) {'y': 2, 'x': 1}
*
**
Unpacking arguments *
>>> ... >>> >>> >>> 1 2
def func(a, b, c, d): print(a, b, c, d) args = (1, 2) args += (3, 4) func(*args) 3 4
** >>> >>> >>> 1 2
args = {'a': 1, 'b': 2, 'c': 3} args['d'] = 4 func(**args) 3 4
>>> func(*(1, 2), **{'d': 4, 'c': 4}) 1 2 4 4 >>> func(1, *(2, 3), **{'d': 4}) 1 2 3 4 >>> func(1, c=3, *(2,), **{'d': 4})
Special Argument-Matching Modes | 447
www.it-ebooks.info
1 2 3 4 >>> func(1, *(2, 3), d=4) 1 2 3 4 >>> f(1, *(2,), c=3, **{'d':4}) 1 2 3 4
* **
*pargs
* func(*open('fname') *pargs def * x, *y = z
Applying functions generically
if if : action, args = func1, (1,) else: action, args = func2, (1, 2, 3) ... action(*args)
448 | Chapter 18: Arguments
www.it-ebooks.info
>>> >>> >>> (2, >>>
args = (2,3) args += (4,) args 3, 4) func(*args)
def tracer(func, *pargs, **kargs): print('calling:', func.__name__) return func(*pargs, **kargs) def func(a, b, c, d): return a + b + c + d print(tracer(func, 1, 2, c=3, d=4))
calling: func 10
The defunct apply built-in (Python 2.6) *args apply
**args
func(*pargs, **kargs) apply(func, pargs, kargs)
>>> def echo(*args, **kwargs): print(args, kwargs) ... >>> echo(1, 2, a=3, b=4) (1, 2) {'a': 3, 'b': 4}
Special Argument-Matching Modes | 449
www.it-ebooks.info
apply >>> pargs = (1, 2) >>> kargs = {'a':3, 'b':4} >>> apply(echo, pargs, kargs) (1, 2) {'a': 3, 'b': 4} >>> echo(*pargs, **kargs) (1, 2) {'a': 3, 'b': 4}
apply *pargs
**kargs
def
>>> echo(0, c=5, *pargs, **kargs) (0, 1, 2) {'a': 3, 'c': 5, 'b': 4}
# Normal, keyword, *sequence, **dictionary
apply
Python 3.0 Keyword-Only Arguments
*args a
b
c >>> def kwonly(a, *b, c): ... print(a, b, c) ... >>> kwonly(1, 2, c=3) 1 (2,) 3 >>> kwonly(a=1, c=3) 1 () 3 >>> kwonly(1, 2, 3) TypeError: kwonly() needs keyword-only argument c
* *
a b
c
450 | Chapter 18: Arguments
www.it-ebooks.info
>>> def kwonly(a, *, b, c): ... print(a, b, c) ... >>> kwonly(1, c=3, b=2) 1 2 3 >>> kwonly(c=3, b=2, a=1) 1 2 3 >>> kwonly(1, 2, 3) TypeError: kwonly() takes exactly 1 positional argument (3 given) >>> kwonly(1) TypeError: kwonly() needs keyword-only argument b
* b
a c
>>> def kwonly(a, *, b='spam', c='ham'): ... print(a, b, c) ... >>> kwonly(1) 1 spam ham >>> kwonly(1, c=3) 1 spam 3 >>> kwonly(a=1) 1 spam ham >>> kwonly(c=3, b=2, a=1) 1 2 3 >>> kwonly(1, 2) TypeError: kwonly() takes exactly 1 positional argument (2 given)
>>> def kwonly(a, *, b, c='spam'): ... print(a, b, c) ... >>> kwonly(1, b='eggs') 1 eggs spam >>> kwonly(1, c='eggs') TypeError: kwonly() needs keyword-only argument b >>> kwonly(1, 2) TypeError: kwonly() takes exactly 1 positional argument (2 given) >>> def kwonly(a, *, b=1, c, d=2): ... print(a, b, c, d) ... >>> kwonly(3, c=4) 3 1 4 2 >>> kwonly(3, c=4, b=5) 3 5 4 2 >>> kwonly(3) TypeError: kwonly() needs keyword-only argument c >>> kwonly(1, 2, 3) TypeError: kwonly() takes exactly 1 positional argument (3 given)
Special Argument-Matching Modes | 451
www.it-ebooks.info
Ordering rules **args ** >>> def kwonly(a, **pargs, b, c): SyntaxError: invalid syntax >>> def kwonly(a, **, b, c): SyntaxError: invalid syntax
**args
*args *args
>>> def f(a, *b, **d, c=6): print(a, b, c, d) SyntaxError: invalid syntax >>> def f(a, *b, c=6, **d): print(a, b, c, d) ... >>> f(1, 2, 3, x=4, y=5) 1 (2, 3) 6 {'y': 5, 'x': 4} >>> f(1, 2, 3, x=4, y=5, c=7) 1 (2, 3) 7 {'y': 5, 'x': 4} >>> f(1, 2, 3, c=7, x=4, y=5) 1 (2, 3) 7 {'y': 5, 'x': 4} >>> def f(a, c=6, *b, **d): print(a, b, c, d) ... >>> f(1, 2, 3, x=4) 1 (3,) 2 {'x': 4}
**args *args >>> def f(a, *b, c=6, **d): print(a, b, c, d) ... >>> f(1, *(2, 3), **dict(x=4, y=5)) 1 (2, 3) 6 {'y': 5, 'x': 4} >>> f(1, *(2, 3), **dict(x=4, y=5), c=7) SyntaxError: invalid syntax >>> f(1, *(2, 3), c=7, **dict(x=4, y=5)) 1 (2, 3) 7 {'y': 5, 'x': 4} >>> f(1, c=7, *(2, 3), **dict(x=4, y=5)) 1 (2, 3) 7 {'y': 5, 'x': 4} >>> f(1, *(2, 3), **dict(x=4, y=5, c=7)) 1 (2, 3) 7 {'y': 5, 'x': 4}
452 | Chapter 18: Arguments
**args
www.it-ebooks.info
Why keyword-only arguments?
process(X, Y, Z) process(X, Y, notify=True)
# use flag's default # override flag default
*args
**args
notify def process(*args, notify=False): ...
The min Wakeup Call!
None * for
The min Wakeup Call! | 453
www.it-ebooks.info
Full Credit args
list sort sort
def min1(*args): res = args[0] for arg in args[1:]: if arg < res: res = arg return res def min2(first, *rest): for arg in rest: if arg < first: first = arg return first def min3(*args): tmp = list(args) tmp.sort() return tmp[0] print(min1(3,4,1,2))
sort
time timeit
454 | Chapter 18: Arguments
www.it-ebooks.info
print(min2("bb", "aa")) print(min3([2,2], [1,1], [3,3]))
% python mins.py 1 aa [1, 1]
Bonus Points <
>
tmp[−1]
tmp[0]
eval def minmax(test, *args): res = args[0] for arg in args[1:]: if test(arg, res): res = arg return res def lessthan(x, y): return x < y def grtrthan(x, y): return x > y print(minmax(lessthan, 4, 2, 1, 5, 6, 3)) print(minmax(grtrthan, 4, 2, 1, 5, 6, 3)) % python minmax.py
The min Wakeup Call! | 455
www.it-ebooks.info
1 6
max test
The Punch Line... min max
Generalized Set Functions
*args for
def intersect(*args): res = [] for x in args[0]: for other in args[1:]: if x not in other: break else: res.append(x) return res def union(*args): res = [] for seq in args: for x in seq: if not x in res: res.append(x) return res
456 | Chapter 18: Arguments
union
www.it-ebooks.info
args intersect % python >>> from inter2 import intersect, union >>> s1, s2, s3 = "SPAM", "SCAM", "SLAM" >>> intersect(s1, s2), union(s1, s2) (['S', 'A', 'M'], ['S', 'P', 'A', 'M', 'C']) >>> intersect([1,2,3], (1,4)) [1] >>> intersect(s1, s2, s3) ['S', 'A', 'M'] >>> union(s1, s2, s3) ['S', 'P', 'A', 'M', 'C', 'L']
Emulating the Python 3.0 print Function *args **args print print from __future__ import print_function
Emulating the Python 3.0 print Function | 457
www.it-ebooks.info
""" Emulate most of the 3.0 print function for use in 2.X call signature: print30(*args, sep=' ', end='\n', file=None) """ import sys def print30(*args, **kargs): sep = kargs.get('sep', ' ') end = kargs.get('end', '\n') file = kargs.get('file', sys.stdout) output = '' first = True for arg in args: output += ('' if first else sep) + str(arg) first = False file.write(output + end)
print from print30 import print30 print30(1, 2, 3) print30(1, 2, 3, sep='') print30(1, 2, 3, sep='...') print30(1, [2], (3,), sep='...') print30(4, 5, 6, sep='', end='') print30(7, 8, 9) print30() import sys print30(1, 2, 3, sep='??', end='.\n', file=sys.stderr)
print C:\misc> c:\python26\python testprint30.py 1 2 3 123 1...2...3 1...[2]...(3,) 4567 8 9 1??2??3.
458 | Chapter 18: Arguments
www.it-ebooks.info
Using Keyword-Only Arguments
def print30(*args, sep=' ', end='\n', file=sys.stdout): output = '' first = True for arg in args: output += ('' if first else sep) + str(arg) first = False file.write(output + end)
>>> print30(99, name='bob') TypeError: print30() got an unexpected keyword argument 'name'
name dict.pop()
def print30(*args, **kargs): sep = kargs.pop('sep', ' ') end = kargs.pop('end', '\n') file = kargs.pop('file', sys.stdout) if kargs: raise TypeError('extra keywords: %s' % kargs) output = '' first = True for arg in args: output += ('' if first else sep) + str(arg) first = False file.write(output + end)
>>> print30(99, name='bob') TypeError: extra keywords: {'name': 'bob'}
Emulating the Python 3.0 print Function | 459
www.it-ebooks.info
Why You Will Care: Keyword Arguments
tkinter Tkinter tkinter
from tkinter import * widget = Button(text="Press me", command=someFunction)
text command
sorted sorted(iterable, key=None, reverse=False)
None
False
Chapter Summary
460 | Chapter 18: Arguments
www.it-ebooks.info
lambda map
filter
Test Your Knowledge: Quiz >>> def func(a, b=4, c=5): ... print(a, b, c) ... >>> func(1, 2)
>>> def func(a, b, c=5): ... print(a, b, c) ... >>> func(1, c=3, b=2)
>>> def func(a, *pargs): ... print(a, pargs) ... >>> func(1, 2, 3)
>>> def func(a, **kargs): ... print(a, kargs) ... >>> func(a=1, c=3, b=2)
>>> def func(a, b, c=3, d=4): print(a, b, c, d) ... >>> func(1, *(5,6))
Test Your Knowledge: Quiz | 461
www.it-ebooks.info
Test Your Knowledge: Answers '1 2 5' c
1
2
'1 2 3' 1 2
a
b
5 a
b
c
3 '1 (2, 3)'
1
a
*pargs for arg in
pargs: ... '1, {'c': 3, 'b': 2}'
1
a
**kargs for key in kargs: ... *name
'1 5 6 4' 1 6 c
a
5 d
return
return
462 | Chapter 18: Arguments
6 4
b
c
www.it-ebooks.info
CHAPTER 19
Advanced Function Topics
lambda map
filter
lambda
Function Design Concepts
return return
463
www.it-ebooks.info
return
self self.name='bob'
464 | Chapter 19: Advanced Function Topics
www.it-ebooks.info
Recursive Functions
Summation with Recursion sum >>> def mysum(L): ... if not L: ... return 0 ... else: ... return L[0] + mysum(L[1:]) >>> mysum([1, 2, 3, 4, 5]) 15
Recursive Functions | 465
www.it-ebooks.info
L print >>> ... ... ... ... ... ... >>> [1, [2, [3, [4, [5] [] 15
L def mysum(L): print(L) if not L: return 0 else: return L[0] + mysum(L[1:]) mysum([1, 2, 3, 4, 5]) 2, 3, 4, 5] 3, 4, 5] 4, 5] 5]
Coding Alternatives if else
def mysum(L): return 0 if not L else L[0] + mysum(L[1:]) def mysum(L): return L[0] if len(L) == 1 else L[0] + mysum(L[1:]) def mysum(L): first, *rest = L return first if not rest else first + mysum(rest)
+ >>> mysum([1]) 1 >>> mysum([1, 2, 3, 4, 5]) 15 >>> mysum(('s', 'p', 'a', 'm')) 'spam'
466 | Chapter 19: Advanced Function Topics
www.it-ebooks.info
>>> mysum(['spam', 'ham', 'eggs']) 'spamhameggs'
mysum ('spam')) def mysum(first, * rest)
>>> def mysum(L): ... if not L: return 0 ... return nonempty(L) ... >>> def nonempty(L): ... return L[0] + mysum(L[1:]) ... >>> mysum([1.1, 2.2, 3.3, 4.4]) 11.0
Loop Statements Versus Recursion
while
>>> >>> >>> ... ... ... >>> 15
L = [1, 2, 3, 4, 5] sum = 0 while L: sum += L[0] L = L[1:] sum
for
>>> >>> >>> ... >>> 15
L = [1, 2, 3, 4, 5] sum = 0 for x in L: sum += x sum
Recursive Functions | 467
www.it-ebooks.info
Handling Arbitrary Structures
[1, [2, [3, 4], 5], 6, [7, 8]]
def sumtree(L): tot = 0 for x in L: if not isinstance(x, list): tot += x else: tot += sumtree(x) return tot L = [1, [2, [3, 4], 5], 6, [7, 8]] print(sumtree(L))
print(sumtree([1, [2, [3, [4, [5]]]]])) print(sumtree([[[[[1], 2], 3], 4], 5]))
468 | Chapter 19: Advanced Function Topics
www.it-ebooks.info
__setattr__
__getattribute__
Function Objects: Attributes and Annotations
Indirect Function Calls
def =
def
>>> def echo(message): ... print(message) ... >>> echo('Direct call') Direct call >>> x = echo >>> x('Indirect call!') Indirect call!
Function Objects: Attributes and Annotations | 469
www.it-ebooks.info
>>> def indirect(func, arg): ... func(arg) ... >>> indirect(echo, 'Argument call!') Argument call!
>>> schedule = [ (echo, 'Spam!'), (echo, 'Ham!') ] >>> for (func, arg) in schedule: ... func(arg) ... Spam! Ham!
schedule
echo for
>>> def make(label): ... def echo(message): ... print(label + ':' + message) ... return echo ... >>> F = make('Spam') >>> F('Ham!') Spam:Ham! >>> F('Eggs!') Spam:Eggs!
Function Introspection
>>> def func(a): ... b = 'spam' ... return b * a ... >>> func(8) 'spamspamspamspamspamspamspamspam'
470 | Chapter 19: Advanced Function Topics
www.it-ebooks.info
>>> func.__name__ 'func' >>> dir(func) ['__annotations__', '__call__', '__class__', '__closure__', '__code__', ...more omitted... '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']
>>> func.__code__ >>> dir(func.__code__) ['__class__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', ...more omitted... 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_kwonlyargcount', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames'] >>> func.__code__.co_varnames ('a', 'b') >>> func.__code__.co_argcount 1
Function Attributes
>>> func >>> func.count = 0 >>> func.count += 1 >>> func.count 1 >>> func.handles = 'Button-Press' >>> func.handles 'Button-Press' >>> dir(func) ['__annotations__', '__call__', '__class__', '__closure__', '__code__', ...more omitted... __str__', '__subclasshook__', 'count', 'handles']
Function Objects: Attributes and Annotations | 471
www.it-ebooks.info
Function Annotations in 3.0
__annotations__
>>> def func(a, b, c): ... return a + b + c ... >>> func(1, 2, 3) 6
def
-> >>> def func(a: 'spam', b: (1, 10), c: float) -> int: ... return a + b + c ... >>> func(1, 2, 3) 6
>>> func.__annotations__ {'a': 'spam', 'c': , 'b': (1, 10), 'return': }
472 | Chapter 19: Advanced Function Topics
www.it-ebooks.info
>>> def func(a: 'spam', b, c: 99): ... return a + b + c ... >>> func(1, 2, 3) 6 >>> func.__annotations__ {'a': 'spam', 'c': 99} >>> for arg in func.__annotations__: ... print(arg, '=>', func.__annotations__[arg]) ... a => spam c => 99
: = a
4
a: 'spam' = 4 'spam'
>>> def func(a: 'spam' = 4, b: (1, 10) = 5, c: float = 6) -> int: ... return a + b + c ... >>> func(1, 2, 3) 6 >>> func() 15 >>> func(1, c=10) 16 >>> func.__annotations__ {'a': 'spam', 'c': , 'b': (1, 10), 'return': }
>>> def func(a:'spam'=4, b:(1,10)=5, c:float=6)->int: ... return a + b + c ... >>> func(1, 2) 9 >>> func.__annotations__ {'a': 'spam', 'c': , 'b': (1, 10), 'return': }
Function Objects: Attributes and Annotations | 473
www.it-ebooks.info
def
lambda
lambda
Anonymous Functions: lambda def lambda
def lambda
lambda Basics lambda
lambda def
lambda argument1, argument2,... argumentN :expression using arguments
lambda def
lambda
lambda
lambda def lambda def
lambda
lambda def lambda lambda
def if lambda
def
lambda
lambda
474 | Chapter 19: Advanced Function Topics
return
www.it-ebooks.info
def
lambda def
>>> def func(x, y, z): return x + y + z ... >>> func(2, 3, 4) 9
lambda >>> f = lambda x, y, z: x + y + z >>> f(2, 3, 4) 9
f
lambda lambda
def
def
>>> x = (lambda a="fee", b="fie", c="foe": a + b + c) >>> x("wee") 'weefiefoe'
lambda def lambda
def
>>> def knights(): ... title = 'Sir' ... action = (lambda x: title + ' ' + x) ... return action ... >>> act = knights() >>> act('robin') 'Sir robin'
title
Why Use lambda? lambda def
lambda def
Anonymous Functions: lambda | 475
www.it-ebooks.info
lambda L = [lambda x: x ** 2, lambda x: x ** 3, lambda x: x ** 4] for f in L: print(f(2)) print(L[0](3))
lambda
def
lambda
def def
def f1(x): return x ** 2 def f2(x): return x ** 3 def f3(x): return x ** 4 L = [f1, f2, f3] for f in L: print(f(2)) print(L[0](3))
>>> key = 'got' >>> {'already': (lambda: 2 + 2), ... 'got': (lambda: 2 * 4), ... 'one': (lambda: 2 ** 6)}[key]() 8
lambda
if lambda
>>> def f1(): return 2 + 2 ... >>> def f2(): return 2 * 4 ...
476 | Chapter 19: Advanced Function Topics
def
www.it-ebooks.info
>>> def f3(): return 2 ** 6 ... >>> key = 'one' >>> {'already': f1, 'got': f2, 'one': f3}[key]() 64
def lambda
lambda
def
lambda map
How (Not) to Obfuscate Your Python Code lambda lambda lambda sys.stdout.write(str(x)+'\n') print
print(x) lambda
if else and or
if a: b else: c
b if a else c ((a and b) or c)
lambda lambda >>> lower = (lambda x, y: x if x < y else y) >>> lower('bb', 'aa') 'aa' >>> lower('aa', 'bb') 'aa'
Anonymous Functions: lambda | 477
www.it-ebooks.info
lambda map >>> import sys >>> showall = lambda x: list(map(sys.stdout.write, x)) >>> t = showall(['spam\n', 'toast\n', 'eggs\n']) spam toast eggs >>> showall = lambda x: [sys.stdout.write(line) for line in x] >>> t = showall(('bright\n', 'side\n', 'of\n', 'life\n')) bright side of life
lambda def lambda
Nested lambdas and Scopes lambda lambda x
def >>> def action(x): ... return (lambda y: x + y) ... >>> act = action(99) >>> act >>> act(2) 101
lambda def >>> action = (lambda x: (lambda y: x + y)) >>> act = action(99) >>> act(3) 102
478 | Chapter 19: Advanced Function Topics
lambda lambda
www.it-ebooks.info
>>> ((lambda x: (lambda y: x + y))(99))(4) 103
lambda lambda
x
lambda lambda
Why You Will Care: Callbacks lambda tkinter
Tkinter
tkinter import sys from tkinter import Button, mainloop # Tkinter in 2.6 x = Button( text ='Press me', command=(lambda:sys.stdout.write('Spam\n'))) x.pack() mainloop()
lambda command
lambda
lambda
def write
lambda
self class MyGui: def makewidgets(self): Button(command=(lambda: self.onPress("spam"))) def onPress(self, message): ...use message...
self
lambda
Mapping Functions over Sequences: map for Mapping Functions over Sequences: map | 479
www.it-ebooks.info
>>> counters = [1, 2, 3, 4] >>> >>> updated = [] >>> for x in counters: ... updated.append(x + 10) ... >>> updated [11, 12, 13, 14]
map
>>> def inc(x): return x + 10 ... >>> list(map(inc, counters)) [11, 12, 13, 14]
map map
inc map
list map lambda >>> list(map((lambda x: x + 3), counters)) [4, 5, 6, 7]
counters lambda for >>> def mymap(func, seq): ... res = [] ... for x in seq: res.append(func(x)) ... return res
inc >>> list(map(inc, [1, 2, 3])) [11, 12, 13] >>> mymap(inc, [1, 2, 3]) [11, 12, 13]
map for
480 | Chapter 19: Advanced Function Topics
map
map
www.it-ebooks.info
>>> pow(3, 4) 81 >>> list(map(pow, [1, 2, 3], [2, 3, 4])) [1, 8, 81]
map pow map
map map map
Functional Programming Tools: filter and reduce map filter reduce filter
range
list filter
>>> list(range(−5, 5)) [−5, −4, −3, −2, −1, 0, 1, 2, 3, 4] >>> list(filter((lambda x: x > 0), range(−5, 5))) [1, 2, 3, 4]
map >>> >>> ... ... ... >>> [1,
reduce
for
res = [] for x in range(−5, 5): if x > 0: res.append(x) res 2, 3, 4]
functools
Functional Programming Tools: filter and reduce | 481
www.it-ebooks.info
reduce >>> from functools import reduce >>> reduce((lambda x, y: x + y), [1, 2, 3, 4]) 10 >>> reduce((lambda x, y: x * y), [1, 2, 3, 4]) 24
reduce lambda for >>> >>> >>> ... ... >>> 10
L = [1,2,3,4] res = L[0] for x in L[1:]: res = res + x res
reduce
>>> def myreduce(function, sequence): ... tally = sequence[0] ... for next in sequence[1:]: ... tally = function(tally, next) ... return tally ... >>> myreduce((lambda x, y: x + y), [1, 2, 3, 4, 5]) 15 >>> myreduce((lambda x, y: x * y), [1, 2, 3, 4, 5]) 120
reduce
operator
>>> import operator, functools >>> functools.reduce(operator.add, [2, 4, 6]) 12 >>> functools.reduce((lambda x, y: x + y), [2, 4, 6]) 12
482 | Chapter 19: Advanced Function Topics
www.it-ebooks.info
map filter
reduce
lambda
Chapter Summary lambda filter
map
reduce
Test Your Knowledge: Quiz lambda
def lamba map filter
reduce
Test Your Knowledge: Answers lambda
def
lambda def
lambda
lambda lambda
def lambda
map
filter
Test Your Knowledge: Answers | 483
www.it-ebooks.info
map filter
True
reduce reduce
__annotations__
484 | Chapter 19: Advanced Function Topics
functools
www.it-ebooks.info
CHAPTER 20
Iterations and Comprehensions, Part 2
map
filter
map
filter
for
List Comprehensions Revisited: Functional Tools
map
map
filter
filter
485
www.it-ebooks.info
List Comprehensions Versus map ord chr >>> ord('s') 115
for >>> res = [] >>> for x in 'spam': ... res.append(ord(x)) ... >>> res [115, 112, 97, 109]
map >>> res = list(map(ord, 'spam')) >>> res [115, 112, 97, 109]
map >>> res = [ord(x) for x in 'spam'] >>> res [115, 112, 97, 109]
for for map >>> [x ** 2 for x in range(10)] [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
map
486 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
lambda def >>> list(map((lambda x: x ** 2), range(10))) [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
lambda
Adding Tests and Nested Loops: filter if
for
if
filter if
map
filter lambda
for >>> [x for x in range(5) if x % 2 == 0] [0, 2, 4] >>> list(filter((lambda x: x % 2 == 0), range(5))) [0, 2, 4] >>> >>> ... ... ... >>> [0,
res = [] for x in range(5): if x % 2 == 0: res.append(x) res 2, 4]
% filter if filter
map
>>> [x ** 2 for x in range(10) if x % 2 == 0] [0, 4, 16, 36, 64]
for if map
List Comprehensions Revisited: Functional Tools | 487
www.it-ebooks.info
filter
map
>>> list( map((lambda x: x**2), filter((lambda x: x % 2 == 0), range(10))) ) [0, 4, 16, 36, 64]
for
if [ expression for target1 in iterable1 [if condition1] for target2 in iterable2 [if condition2] ... for targetN in iterableN [if conditionN] ]
for for >>> res = [x + y for x in [0, 1, 2] for y in [100, 200, 300]] >>> res [100, 200, 300, 101, 201, 301, 102, 202, 302]
>>> res = [] >>> for x in [0, 1, 2]: ... for y in [100, 200, 300]: ... res.append(x + y) ... >>> res [100, 200, 300, 101, 201, 301, 102, 202, 302]
>>> [x + y for x in 'spam' for y in 'SPAM'] ['sS', 'sP', 'sA', 'sM', 'pS', 'pP', 'pA', 'pM', 'aS', 'aP', 'aA', 'aM', 'mS', 'mP', 'mA', 'mM']
if
for
>>> [(x, y) for x in range(5) if x % 2 == 0 for y in range(5) if y % 2 == 1] [(0, 1), (0, 3), (2, 1), (2, 3), (4, 1), (4, 3)]
if >>> res = [] >>> for x in range(5): ... if x % 2 == 0: ... for y in range(5): ... if y % 2 == 1: ... res.append((x, y)) ...
488 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
>>> res [(0, 1), (0, 3), (2, 1), (2, 3), (4, 1), (4, 3)]
for
map
if
filter
List Comprehensions and Matrixes
>>> M = [[1, 2, 3], ... [4, 5, 6], ... [7, 8, 9]] >>> N = [[2, 2, 2], ... [3, 3, 3], ... [4, 4, 4]]
>>> M[1] [4, 5, 6] >>> M[1][2] 6
>>> [row[1] for row in M] [2, 5, 8] >>> [M[row][1] for row in (0, 1, 2)] [2, 5, 8]
range M[0][0]
M[1][1]
List Comprehensions Revisited: Functional Tools | 489
www.it-ebooks.info
>>> [M[i][i] for i in range(len(M))] [1, 5, 9]
>>> [M[row][col] * N[row][col] for row in range(3) for col in range(3)] [2, 4, 6, 12, 15, 18, 28, 32, 36] >>> [[M[row][col] * N[row][col] for col in range(3)] for row in range(3)] [[2, 4, 6], [12, 15, 18], [28, 32, 36]]
>>> res >>> for ... ... ... ... ... >>> res [[2, 4,
= [] row in range(3): tmp = [] for col in range(3): tmp.append(M[row][col] * N[row][col]) res.append(tmp) 6], [12, 15, 18], [28, 32, 36]]
Comprehending List Comprehensions for map
map for map
map
for for map
map time
time.clock
time.time
490 | Chapter 20: Iterations and Comprehensions, Part 2
timeit
www.it-ebooks.info
for for map map for lambda map
Why You Will Care: List Comprehensions and map map map
readlines
\n
>>> open('myfile').readlines() ['aaa\n', 'bbb\n', 'ccc\n']
map
map
list >>> [line.rstrip() for line in open('myfile').readlines()] ['aaa', 'bbb', 'ccc'] >>> [line.rstrip() for line in open('myfile')] ['aaa', 'bbb', 'ccc'] >>> list(map((lambda line: line.rstrip()), open('myfile'))) ['aaa', 'bbb', 'ccc']
map
listoftuple = [('bob', 35, 'mgr'), ('mel', 40, 'dev')]
for
map
>>> [age for (name, age, job) in listoftuple] [35, 40] >>> list(map((lambda row: row[1]), listoftuple)) [35, 40]
List Comprehensions Revisited: Functional Tools | 491
www.it-ebooks.info
map >>> list(map((lambda (name, age, job): age), listoftuple)) [35, 40]
map
map
Iterators Revisited: Generators
def
Generator Functions: yield Versus return
def
492 | Chapter 20: Iterations and Comprehensions, Part 2
yield
www.it-ebooks.info
State suspension
yield
yield
Iteration protocol integration
__next__ StopIteration iter for
yield __next__ return def StopIteration __next__ yield
StopIteration def
yield
Iterators Revisited: Generators | 493
www.it-ebooks.info
next
__next__ __next__
next next(I)
I.__next__() I.next()
I.next()
Generator functions in action
>>> def gensquares(N): ... for i in range(N): ... yield i ** 2 ...
yield
for yield
>>> for i in gensquares(5): ... print(i, end=' : ') ... 0 : 1 : 4 : 9 : 16 : >>>
return for >>> x = gensquares(4) >>> x
__next__ StopIteration next(X) X.__next__() >>> 0 >>> 1 >>> 4 >>> 9 >>>
next(x) next(x) next(x) next(x) next(x)
494 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
Traceback (most recent call last): ...more text omitted... StopIteration
for __next__ for
>>> def buildsquares(n): ... res = [] ... for i in range(n): res.append(i ** 2) ... return res ... >>> for x in buildsquares(5): print(x, end=' : ') ... 0 : 1 : 4 : 9 : 16 :
for
map
>>> for x in [n ** 2 for n in range(5)]: ... print(x, end=' : ') ... 0 : 1 : 4 : 9 : 16 : >>> for x in map((lambda n: n ** 2), range(5)): ... print(x, end=' : ') ... 0 : 1 : 4 : 9 : 16 :
yield
yield next
Iterators Revisited: Generators | 495
www.it-ebooks.info
Extended generator function protocol: send versus next send
send __next__
yield yield X X = yield Y
send A = (yield X) X = (yield Y) + 42 G
G.send(value) send next(G)
yield
yield G.__next__() None
>>> def gen(): ... for i in range(10): ... X = yield i ... print(X) ... >>> G = gen() >>> next(G) 0 >>> G.send(77) 77 1 >>> G.send(88) 88 2 >>> next(G) None 3
send
yield
throw(type) close
Generator
Exit
next(X) X.__next__()
send G.send(X)
__next__
496 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
Generator Expressions: Iterators Meet Comprehensions
>>> [x ** 2 for x in range(4)] [0, 1, 4, 9] >>> (x ** 2 for x in range(4))
list >>> list(x ** 2 for x in range(4)) [0, 1, 4, 9]
>>> >>> 0 >>> 1 >>> 4 >>> 9 >>>
G = (x ** 2 for x in range(4)) next(G) next(G) next(G) next(G) next(G)
Traceback (most recent call last): ...more text omitted... StopIteration
next for >>> for num in (x ** 2 for x in range(4)): ... print('%s, %s' % (num, num / 2.0)) ... 0, 0.0 1, 0.5 4, 2.0 9, 4.5
sum map sorted any all
list
Iterators Revisited: Generators | 497
www.it-ebooks.info
sorted >>> sum(x ** 2 for x in range(4)) 14 >>> sorted(x ** 2 for x in range(4)) [0, 1, 4, 9] >>> sorted((x ** 2 for x in range(4)), reverse=True) [9, 4, 1, 0] >>> import math >>> list( map(math.sqrt, (x ** 2 for x in range(4))) ) [0.0, 1.0, 2.0, 3.0]
Generator Functions Versus Generator Expressions
>>> G = (c * 4 for c in 'SPAM') >>> list(G) ['SSSS', 'PPPP', 'AAAA', 'MMMM']
>>> def timesfour(S): ... for c in S: ... yield c * 4 ... >>> G = timesfour('spam') >>> list(G) ['ssss', 'pppp', 'aaaa', 'mmmm']
list >>> G = (c * 4 for c in 'SPAM') >>> I = iter(G) >>> next(I) 'SSSS' >>> next(I) 'PPPP'
498 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
>>> G = timesfour('spam') >>> I = iter(G) >>> next(I) 'ssss' >>> next(I) 'pppp'
Generators Are Single-Iterator Objects
iter >>> G = (c * 4 for c in 'SPAM') >>> iter(G) is G True
>>> G = (c * 4 for c in 'SPAM') >>> I1 = iter(G) >>> next(I1) 'SSSS' >>> next(I1) 'PPPP' >>> I2 = iter(G) >>> next(I2) 'AAAA'
>>> list(I1) ['MMMM'] >>> next(I2) StopIteration >>> I3 = iter(G) >>> next(I3) StopIteration >>> I3 = iter(c * 4 for c in 'SPAM') >>> next(I3) 'SSSS'
Iterators Revisited: Generators | 499
www.it-ebooks.info
def >>> def timesfour(S): ... for c in S: ... yield c * 4 ... >>> G = timesfour('spam') >>> iter(G) is G True >>> I1, I2 = iter(G), iter(G) >>> next(I1) 'ssss' >>> next(I1) 'pppp' >>> next(I2) 'aaaa'
>>> L = [1, 2, 3, 4] >>> I1, I2 = iter(L), iter(L) >>> next(I1) 1 >>> next(I1) 2 >>> next(I2) 1 >>> del L[2:] >>> next(I1) StopIteration
Emulating zip and map with Iteration Tools
zip map zip >>> S1 = 'abc' >>> S2 = 'xyz123' >>> list(zip(S1, S2)) [('a', 'x'), ('b', 'y'), ('c', 'z')]
500 | Chapter 20: Iterations and Comprehensions, Part 2
map
www.it-ebooks.info
>>> list(zip([−2, −1, 0, 1, 2])) [(−2,), (−1,), (0,), (1,), (2,)] >>> list(zip([1, 2, 3], [2, 3, 4, 5])) [(1, 2), (2, 3), (3, 4)]
>>> list(map(abs, [−2, −1, 0, 1, 2])) [2, 1, 0, 1, 2] >>> list(map(pow, [1, 2, 3], [2, 3, 4, 5])) [1, 8, 81]
zip
Coding your own map(func, ...) map
zip map
def mymap(func, *seqs): res = [] for args in zip(*seqs): res.append(func(*args)) return res print(mymap(abs, [−2, −1, 0, 1, 2])) print(mymap(pow, [1, 2, 3], [2, 3, 4, 5]))
*args zip zip
map [2, 1, 0, 1, 2] [1, 8, 81]
for
Iterators Revisited: Generators | 501
www.it-ebooks.info
def mymap(func, *seqs): return [func(*args) for args in zip(*seqs)] print(mymap(abs, [−2, −1, 0, 1, 2])) print(mymap(pow, [1, 2, 3], [2, 3, 4, 5]))
mymap
def mymap(func, *seqs): res = [] for args in zip(*seqs): yield func(*args) def mymap(func, *seqs): return (func(*args) for args in zip(*seqs))
list print(list(mymap(abs, [−2, −1, 0, 1, 2]))) print(list(mymap(pow, [1, 2, 3], [2, 3, 4, 5])))
list zip
Coding your own zip(...) and map(None, ...) zip map map map
None
C:\misc> c:\python26\python >>> map(None, [1, 2, 3], [2, 3, 4, 5]) [(1, 2), (2, 3), (3, 4), (None, 5)] >>> map(None, 'abc', 'xyz123') [('a', 'x'), ('b', 'y'), ('c', 'z'), (None, '1'), (None, '2'), (None, '3')]
502 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
zip map
def myzip(*seqs): seqs = [list(S) for S in seqs] res = [] while all(seqs): res.append(tuple(S.pop(0) for S in seqs)) return res def mymapPad(*seqs, pad=None): seqs = [list(S) for S in seqs] res = [] while any(seqs): res.append(tuple((S.pop(0) if S else pad) for S in seqs)) return res S1, S2 = 'abc', 'xyz123' print(myzip(S1, S2)) print(mymapPad(S1, S2)) print(mymapPad(S1, S2, pad=99))
list any
all True
True
pad
map
**kargs zip
map
[('a', 'x'), ('b', 'y'), ('c', 'z')] [('a', 'x'), ('b', 'y'), ('c', 'z'), (None, '1'), (None, '2'), (None, '3')] [('a', 'x'), ('b', 'y'), ('c', 'z'), (99, '1'), (99, '2'), (99, '3')]
zip
map yield
list
def myzip(*seqs): seqs = [list(S) for S in seqs] while all(seqs): yield tuple(S.pop(0) for S in seqs)
Iterators Revisited: Generators | 503
www.it-ebooks.info
def mymapPad(*seqs, pad=None): seqs = [list(S) for S in seqs] while any(seqs): yield tuple((S.pop(0) if S else pad) for S in seqs) S1, S2 = 'abc', 'xyz123' print(list(myzip(S1, S2))) print(list(mymapPad(S1, S2))) print(list(mymapPad(S1, S2, pad=99)))
zip
map
pop
def myzip(*seqs): minlen = min(len(S) for S in seqs) return [tuple(S[i] for S in seqs) for i in range(minlen)] def mymapPad(*seqs, pad=None): maxlen = max(len(S) for S in seqs) index = range(maxlen) return [tuple((S[i] if len(S) > i else pad) for S in seqs) for i in index] S1, S2 = 'abc', 'xyz123' print(myzip(S1, S2)) print(mymapPad(S1, S2)) print(mymapPad(S1, S2, pad=99))
len tuple
min
max range tuple
zip
def myzip(*seqs): minlen = min(len(S) for S in seqs)
504 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
return (tuple(S[i] for S in seqs) for i in range(minlen)) print(list(myzip(S1, S2)))
list
Why You Will Care: One-Shot Iterations map
zip def myzip(*args): iters = map(iter, args) while iters: res = [next(i) for i in iters] yield tuple(res)
iter next StopIteration
next(it) return
while iters:
>>> list(myzip('abc', 'lmnop')) [('a', 'l'), ('b', 'm'), ('c', 'n')]
map iters
res
[]
list def myzip(*args): iters = list(map(iter, args)) ...rest as is...
map list
Iterators Revisited: Generators | 505
www.it-ebooks.info
Value Generation in Built-in Types and Classes
>>> >>> >>> 'a' >>> 'c'
D = {'a':1, 'b':2, 'c':3} x = iter(D) next(x) next(x)
for
map
>>> for key in D: ... print(key, D[key]) ... a 1 c 3 b 2
>>> for line in open('temp.txt'): ... print(line, end='') ... Tis but a flesh wound.
for
__iter__ __next__
for
506 | Chapter 20: Iterations and Comprehensions, Part 2
iter next
__getitem__
www.it-ebooks.info
3.0 Comprehension Syntax Summary
{1, 3, 2} set([1, 3, 2]) {f(x) for x in S if P(x)} set(f(x) for x in S if P(x)) f(x) val) in zip(keys, vals)} f(x) for x in items} items)
{key: val for (key, dict(zip(keys, vals)) {x: dict((x, f(x)) for x in
>>> [x * x for x in range(10)] [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] >>> (x * x for x in range(10)) >>> {x * x for x in range(10)} {0, 1, 4, 81, 64, 9, 16, 49, 25, 36}
0
>>> {x: x * x for x in range(10)} {0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
Comprehending Set and Dictionary Comprehensions
>>> {0, >>> {0,
{x * x for x in range(10)} 1, 4, 81, 64, 9, 16, 49, 25, 36} set(x * x for x in range(10)) 1, 4, 81, 64, 9, 16, 49, 25, 36}
>>> {0: >>> {0:
{x: x * x for x in range(10)} 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81} dict((x, x * x) for x in range(10)) 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
3.0 Comprehension Syntax Summary | 507
www.it-ebooks.info
>>> >>> ... ... >>> {0,
res = set() for x in range(10): res.add(x * x)
>>> >>> ... ... >>> {0:
res = {} for x in range(10): res[x] = x * x
>>> >>> (0, >>> (1,
G = ((x, x * x) for x in range(10)) next(G) 0) next(G) 1)
res 1, 4, 81, 64, 9, 16, 49, 25, 36}
res 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
Extended Comprehension Syntax for Sets and Dictionaries if
>>> [0, >>> {0, >>> {0:
[x * x for x in 4, 16, 36, 64] {x * x for x in 16, 4, 64, 36} {x: x * x for x 0, 8: 64, 2: 4,
range(10) if x % 2 == 0] range(10) if x % 2 == 0} in range(10) if x % 2 == 0} 4: 16, 6: 36}
for >>> [5, >>> {8, >>> {1:
[x + y for x in [1, 2, 3] for y in [4, 5, 6]] 6, 7, 6, 7, 8, 7, 8, 9] {x + y for x in [1, 2, 3] for y in [4, 5, 6]} 9, 5, 6, 7} {x: y for x in [1, 2, 3] for y in [4, 5, 6]} 6, 2: 6, 3: 6}
>>> {x + y for x in 'ab' for y in 'cd'} {'bd', 'ac', 'ad', 'bc'}
508 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
>>> {x + y: (ord(x), ord(y)) for x in 'ab' for y in 'cd'} {'bd': (98, 100), 'ac': (97, 99), 'ad': (97, 100), 'bc': (98, 99)} >>> {k * 2 for k in ['spam', 'ham', 'sausage'] if k[0] == 's'} {'sausagesausage', 'spamspam'} >>> {k.upper(): k * 2 for k in ['spam', 'ham', 'sausage'] if k[0] == 's'} {'SAUSAGE': 'sausagesausage', 'SPAM': 'spamspam'}
for
Timing Iteration Alternatives
for
map
Timing Module
import time reps = 1000 repslist = range(reps) def timer(func, *pargs, **kargs): start = time.clock() for i in repslist: ret = func(*pargs, **kargs) elapsed = time.clock() - start return (elapsed, ret)
Timing Iteration Alternatives | 509
www.it-ebooks.info
time
range range reps
Timing Script
import sys, mytimer reps = 10000 repslist = range(reps) def forLoop(): res = [] for x in repslist: res.append(abs(x)) return res def listComp(): return [abs(x) for x in repslist] def mapCall(): return list(map(abs, repslist)) def genExpr(): return list(abs(x) for x in repslist) def genFunc(): def gen():
510 | Chapter 20: Iterations and Comprehensions, Part 2
mytimer.reps = N
www.it-ebooks.info
for x in repslist: yield abs(x) return list(gen()) print(sys.version) for test in (forLoop, listComp, mapCall, genExpr, genFunc): elapsed, result = mytimer.timer(test) print ('-' * 33) print ('%-9s: %.5f => [%s...%s]' % (test.__name__, elapsed, result[0], result[-1]))
list map __name__
Timing Results map for C:\misc> c:\python30\python timeseqs.py 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)] --------------------------------forLoop : 2.64441 => [0...9999] --------------------------------listComp : 1.60110 => [0...9999] --------------------------------mapCall : 1.41977 => [0...9999] --------------------------------genExpr : 2.21758 => [0...9999] --------------------------------genFunc : 2.18696 => [0...9999]
list list return [abs(x) for x in range(size)] return list(abs(x) for x in range(size))
Timing Iteration Alternatives | 511
www.it-ebooks.info
for
map abs
2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] forStatement => 6.10899996758 listComprehension => 3.51499986649 mapFunction => 2.73399996758 generatorExpression => 4.11600017548
list
map
abs
... ... def forLoop(): res = [] for x in repslist: res.append(x + 10) return res def listComp(): return [x + 10 for x in repslist] def mapCall(): return list(map((lambda x: x + 10), repslist)) def genExpr(): return list(x + 10 for x in repslist) def genFunc(): def gen(): for x in repslist: yield x + 10 return list(gen()) ... ...
map for C:\misc> c:\python30\python timeseqs.py 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)]
512 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
--------------------------------forLoop : 2.60754 => [10...10009] --------------------------------listComp : 1.57585 => [10...10009] --------------------------------mapCall : 3.10276 => [10...10009] --------------------------------genExpr : 1.96482 => [10...10009] --------------------------------genFunc : 1.95340 => [10...10009]
2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] forStatement => 5.25699996948 listComprehension => 2.68400001526 mapFunction => 5.96900010109 generatorExpression => 3.37400007248
map
Timing Module Alternatives time.clock time.time timer
Timing Iteration Alternatives | 513
www.it-ebooks.info
_reps
""" timer(spam, 1, 2, a=3, b=4, _reps=1000) calls and times spam(1, 2, a=3) _reps times, and returns total time for all runs, with final result; best(spam, 1, 2, a=3, b=4, _reps=50) runs best-of-N timer to filter out any system load variation, and returns best time among _reps tests """ import time, sys if sys.platform[:3] == 'win': timefunc = time.clock else: timefunc = time.time def trace(*args): pass def timer(func, *pargs, **kargs): _reps = kargs.pop('_reps', 1000) trace(func, pargs, kargs, _reps) repslist = range(_reps) start = timefunc() for i in repslist: ret = func(*pargs, **kargs) elapsed = timefunc() - start return (elapsed, ret) def best(func, *pargs, **kargs): _reps = kargs.pop('_reps', 50) best = 2 ** 32 for i in range(_reps): (time, ret) = timer(func, *pargs, _reps=1, **kargs) if time < best: best = time return (best, ret)
pop
_reps trace
print x + 1
import sys, mytimer reps = 10000 repslist = range(reps) def forLoop(): ...
514 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
def listComp(): ... def mapCall(): ... def genExpr(): ... def genFunc(): ... print(sys.version) for tester in (mytimer.timer, mytimer.best): print('' % tester.__name__) for test in (forLoop, listComp, mapCall, genExpr, genFunc): elapsed, result = tester(test) print ('-' * 35) print ('%-9s: %.5f => [%s...%s]' % (test.__name__, elapsed, result[0], result[-1]))
C:\misc> c:\python30\python timeseqs.py 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)] ----------------------------------forLoop : 2.35371 => [10...10009] ----------------------------------listComp : 1.29640 => [10...10009] ----------------------------------mapCall : 3.16556 => [10...10009] ----------------------------------genExpr : 1.97440 => [10...10009] ----------------------------------genFunc : 1.95072 => [10...10009] ----------------------------------forLoop : 0.00193 => [10...10009] ----------------------------------listComp : 0.00124 => [10...10009] ----------------------------------mapCall : 0.00268 => [10...10009] ----------------------------------genExpr : 0.00164 => [10...10009] ----------------------------------genFunc : 0.00165 => [10...10009]
map
Timing Iteration Alternatives | 515
www.it-ebooks.info
Using keyword-only arguments in 3.0
_reps *
** **
""" Use 3.0 keyword-only default arguments, instead of ** and dict pops. No need to hoist range() out of test in 3.0: a generator, not a list """ import time, sys trace = lambda *args: None # or print timefunc = time.clock if sys.platform == 'win32' else time.time def timer(func, *pargs, _reps=1000, **kargs): trace(func, pargs, kargs, _reps) start = timefunc() for i in range(_reps): ret = func(*pargs, **kargs) elapsed = timefunc() - start return (elapsed, ret) def best(func, *pargs, _reps=50, **kargs): best = 2 ** 32 for i in range(_reps): (time, ret) = timer(func, *pargs, _reps=1, **kargs) if time < best: best = time return (best, ret)
C:\misc> c:\python30\python timeseqs.py ...same results as before...
C:\misc> c:\python30\python >>> from mytimer import timer, best >>> >>> def power(X, Y): return X ** Y ... >>> timer(power, 2, 32) (0.002625403507987747, 4294967296) >>> timer(power, 2, 32, _reps=1000000)
516 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
(1.1822605247314932, 4294967296) >>> timer(power, 2, 100000)[0] 2.2496919999608878 >>> best(power, 2, 32) (5.58730229727189e-06, 4294967296) >>> best(power, 2, 100000)[0] 0.0019937589833460834 >>> best(power, 2, 100000, _reps=500)[0] 0.0019845399345541637
X ** Y
2**100,000 >>> timer(power, 2, 1000000, _reps=1)[0] 0.088112804839710179 >>> timer(power, 2, 1000000, _reps=10)[0] 0.40922470593329763 >>> best(power, 2, 1000000, _reps=1)[0] 0.086550036387279761 >>> best(power, 2, 1000000, _reps=10)[0] 0.029616752967200455 >>> best(power, 2, 1000000, _reps=50)[0] 0.029486918030102061
Other Suggestions timeit
profile
Timing Iteration Alternatives | 517
www.it-ebooks.info
str.format % print print('' % tester.__name__) print(''.format(tester.__name__)) print ('%-9s: %.5f => [%s...%s]' % (test.__name__, elapsed, result[0], result[-1])) print('{0: [{2}...{3}]'.format( test.__name__, elapsed, result[0], result[-1]))
for
for
Function Gotchas
Local Names Are Detected Statically def
518 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
>>> X = 99 >>> def selector(): ... print(X) ... >>> selector() 99
X
X X
>>> def selector(): ... print(X) ... X = 88 ... >>> selector() ...error text omitted... UnboundLocalError: local variable 'X' referenced before assignment
X
X
print X =
def
X
X X X
global >>> def selector(): ... global X ... print(X) ... X = 88 ... >>> selector() 99
X X
>>> X = >>> def ... ... ...
99 selector(): import __main__ print(__main__.X) X = 88
Function Gotchas | 519
www.it-ebooks.info
... print(X) ... >>> selector() 99 88
.X __main__
__main__.X
X
Defaults and Mutable Objects def
def
>>> ... ... ... >>> [2, >>> [1] >>> [1, >>> [1,
def saver(x=[]): x.append(1) print(x) saver([2]) 1] saver() saver() 1] saver() 1, 1]
520 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
def
>>> ... ... ... ... ... >>> [2, >>> [1] >>> [1]
def saver(x=None): if x is None: x = [] x.append(1) print(x) saver([2]) 1] saver() saver()
if x = x or []
or None
x
or or
if [] or []
>>> ... ... ... >>> >>> [1] >>> [1, >>> [1,
def saver(): saver.x.append(1) print(saver.x) saver.x = [] saver() saver() 1] saver() 1, 1]
Function Gotchas | 521
www.it-ebooks.info
Functions Without returns return
yield return None
>>> def proc(x): ... print(x) ... >>> x = proc('testing 123...') testing 123... >>> print(x) None
return None
append None >>> list = [1, 2, 3] >>> list = list.append(4) >>> print(list) None
Enclosing Scope Loop Variables
Chapter Summary
522 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
Test Your Knowledge: Quiz
yield map
Test Your Knowledge: Answers
__next__
def __iter__ yield
Test Your Knowledge: Answers | 523
www.it-ebooks.info
yield yield
next
__next__ return
map map
map
map for
if
filter
Test Your Knowledge: Part IV Exercises
adder adder
adder
S[:0]
524 | Chapter 20: Iterations and Comprehensions, Part 2
S min
type
www.it-ebooks.info
adder def adder(good, bad, ugly)
adder(ugly=1, good=2) adder dict.keys for
while
list
copyDict(dict) keys keys
X[:] addDict(dict1, dict2)
type
def f1(a, b): print(a, b) def f2(a, *b): print(a, b) def f3(a, **b): print(a, b) def f4(a, *b, **c): print(a, b, c) def f5(a, b=2, c=3): print(a, b, c) def f6(a, b=2, *c): print(a, b, c)
>>> f1(1, 2) >>> f1(b=2, a=1) >>> f2(1, 2, 3) >>> f3(1, x=2, y=3) >>> f4(1, 2, 3, x=2, y=3)
Test Your Knowledge: Part IV Exercises | 525
www.it-ebooks.info
>>> f5(1) >>> f5(1, 4) >>> f6(1) >>> f6(1, 3, 4)
x = y // 2 while x > 1: if y % x == 0: print(y, 'has factor', x) break x -= 1 else: print(y, 'is prime')
y //
/
/ 0
1
13 is prime 13.0 is prime 15 has factor 5 15.0 has factor 5.0
[2, 4, 9, 16, 25] map math
math.sqrt(X) X ** .5
for sqrt math
math.sqrt(x)
pow(X, .5)
best
for
526 | Chapter 20: Iterations and Comprehensions, Part 2
www.it-ebooks.info
PART V
Modules
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 21
Modules: The Big Picture
import from imp.reload
__name__
__all__
Why Use Modules?
529
www.it-ebooks.info
Python Program Architecture
530 | Chapter 21: Modules: The Big Picture
www.it-ebooks.info
How to Structure a Program
Imports and Attributes
Python Program Architecture | 531
www.it-ebooks.info
spam def def spam(text): print(text, 'spam')
spam import b b.spam('gumby')
import b import
from import import
import
import spam
b
b.spam spam
b 'gumby' object.attribute
a c
b
b.spam
532 | Chapter 21: Modules: The Big Picture
b
www.it-ebooks.info
Standard Library Modules
How Imports Work
C #include
How Imports Work | 533
www.it-ebooks.info
sys.modules
1. Find It import import import b import c:\dir1\b.py import
2. Compile It (Maybe) import
import import
import
534 | Chapter 21: Modules: The Big Picture
www.it-ebooks.info
__name__
__main__
3. Run It
def
print
def
imp.reload
The Module Search Path
sys.modules sys list(sys.modules.keys())
The Module Search Path | 535
www.it-ebooks.info
PYTHONPATH
sys.path
PYTHONPATH PYTHONPATH PYTHONPATH
PYTHONPATH
PYTHONPATH
536 | Chapter 21: Modules: The Big Picture
www.it-ebooks.info
PYTHONPATH
PYTHONPATH PYTHONPATH
site PYTHONPATH
distutils
Configuring the Search Path PYTHONPATH
The Module Search Path | 537
www.it-ebooks.info
PYTHONPATH c:\pycode\utilities;d:\pycode\package1
c:\pycode\utilities d:\pycode\package1
PYTHONPATH
Search Path Variations
PYTHONPATH
sys.path
The sys.path List sys.path path
from
538 | Chapter 21: Modules: The Big Picture
sys
from . import string
www.it-ebooks.info
sys.path PYTHONPATH
PYTHONPATH C:\users\mark
C:\users
>>> import sys >>> sys.path ['', 'C:\\users', 'C:\\Windows\\system32\\python30.zip', 'c:\\Python30\\DLLs', 'c:\\Python30\\lib', 'c:\\Python30\\lib\\plat-win', 'c:\\Python30', 'C:\\Users\\Mark', 'c:\\Python30\\lib\\site-packages']
sys.path PYTHONPATH
Module File Selection import import
import b
sys.path PYTHONPATH import
sys.path sys.path.append(dirname)
The Module Search Path | 539
www.it-ebooks.info
import b b.attr
b
sys.path
Advanced Module Selection Concepts
sys.path __import__
import
-O
Third-Party Software: distutils distutils
540 | Chapter 21: Modules: The Big Picture
www.it-ebooks.info
distutils distutils
distutils
Chapter Summary import
PYTHONPATH
import
Test Your Knowledge: Quiz PYTHONPATH
Test Your Knowledge: Quiz | 541
www.it-ebooks.info
Test Your Knowledge: Answers
PYTHONPATH
PYTHONPATH PYTHONPATH
def class
542 | Chapter 21: Modules: The Big Picture
www.it-ebooks.info
CHAPTER 22
Module Coding Basics
Module Creation
def printer def printer(x): print(x)
if
import if
543
www.it-ebooks.info
M
import
Module Usage import import from
printer
The import Statement module1
>>> import module1 >>> module1.printer('Hello world!') Hello world!
import module1.printer
544 | Chapter 22: Module Coding Basics
from
www.it-ebooks.info
The from Statement from printer >>> from module1 import printer >>> printer('Hello world!') Hello world!
from from import
The from * Statement from
*
printer >>> from module1 import * >>> printer('Hello world!') Hello world!
import
from
from *
from ...*
Module Usage | 545
www.it-ebooks.info
Imports Happen Only Once
import
from
print('hello') spam = 1
print spam
=
% python >>> import simple hello >>> simple.spam 1
spam >>> simple.spam = 2 >>> import simple >>> simple.spam 2
reload
import and from Are Assignments def import
from if
def import
from import from
546 | Chapter 22: Module Coding Basics
def import
from
www.it-ebooks.info
from
x = 1 y = [1, 2] % python >>> from small import x, y >>> x = 42 >>> y[0] = 42
x
y
y
>>> import small >>> small.x 1 >>> small.y [42, 2]
from
Cross-File Name Changes x x from
x import
% python >>> from small import x, y >>> x = 42 >>> import small >>> small.x = 42
y[0]
Module Usage | 547
www.it-ebooks.info
import and from Equivalence import from
small
from from
from module import name1, name2
import module name1 = module.name1 name2 = module.name2 del module
from from * from module import * from
import
from
Potential Pitfalls of the from Statement from name
module.name import from
from
tkinter from
import module.attr attr from from module import x, y, z from reload 548 | Chapter 22: Module Coding Basics
www.it-ebooks.info
from module import *
from *
import from
from
from * from * from
When import is required import
from
def func(): ...do something...
def func(): ...do something else...
from
from M import func from N import func func()
import
import M, N M.func() N.func()
import
Module Usage | 549
www.it-ebooks.info
Module Namespaces
Files Generate Namespaces
X = 1 X M.X
M X
def
class
=
def __dict__ __dict__ dir __dict__
550 | Chapter 22: Module Coding Basics
dir
dir(M)
www.it-ebooks.info
print('starting to load...') import sys name = 42 def func(): pass class klass: pass print('done loading.')
print >>> import module2 starting to load... done loading.
import >>> module2.sys >>> module2.name 42 >>> module2.func >>> module2.klass
sys name func sys
klass import
__dict__
list
>>> list(module2.__dict__.keys()) ['name', '__builtins__', '__file__', '__package__', 'sys', 'klass', 'func', '__name__', '__doc__']
Module Namespaces | 551
www.it-ebooks.info
__file__ __name__
Attribute Name Qualification
object.attribute module2.sys sys L L.append
append
X X.Y X
module2
X X
X.Y.Z
Imports Versus Scopes
552 | Chapter 22: Module Coding Basics
Y
Y
X
Z
X.Y
www.it-ebooks.info
X X X = 88 def f(): global X X = 99
X X = 11 import moda moda.f() print(X, moda.X)
moda.f
X
moda
X
modb
moda.f % python modb.py 11 99
Namespace Nesting
X = 3
X
mod3
Module Namespaces | 553
www.it-ebooks.info
X = 2 import mod3 print(X, end=' ') print(mod3.X)
X
mod2
X = 1 import mod2 print(X, end=' ') print(mod2.X, end=' ') print(mod2.mod3.X)
mod1 mod2
mod2 mod2.mod3.X mod1
mod3 X
% python mod1.py 2 3 1 2 3
mod3
mod2
mod2
mod1 mod1 mod2 import mod1
mod2.mod3.X
import mod2
mod2.mod3 import
Reloading Modules reload import
554 | Chapter 22: Module Coding Basics
from
mod2.mod3.X
import
www.it-ebooks.info
reload
reload reload
reload
reload imp imp.reload import
from reload
imp
reload Basics import
from
reload reload reload reload
import
reload
import module ...use module.attributes... ... ...
Reloading Modules | 555
www.it-ebooks.info
from imp import reload reload(module) ...use module.attributes...
reload reload
reload
def import import from
from
reload Example reload
message = "First version" def printer(): print(message)
message % python >>> import changer >>> changer.printer() First version
556 | Chapter 22: Module Coding Basics
www.it-ebooks.info
...modify changer.py without stopping Python... % vi changer.py
message
printer
message = "After editing" def printer(): print('reloaded:', message)
reload ...back to the Python interpreter/program... >>> import changer >>> changer.printer() First version >>> from imp import reload >>> reload(changer) >>> changer.printer() reloaded: After editing
reload
Why You Will Care: Module Reloads
Reloading Modules | 557
www.it-ebooks.info
Chapter Summary import reload
from
from reload
from
import
Test Your Knowledge: Quiz from
import
reload import
from from
Test Your Knowledge: Answers
558 | Chapter 22: Module Coding Basics
www.it-ebooks.info
from
import from
name
module.name reload
import
from
from reload from *
Test Your Knowledge: Answers | 559
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 23
Module Packages
from
Package Import Basics import import dir1.dir2.mod
from from dir1.dir2.mod import x
561
www.it-ebooks.info
import dir0\dir1\dir2\mod.py
sys.path import
Packages and Search Path Settings import import
C:\dir1 My Documents.dir2
../dir1
import C:\mycode\dir1\dir2\mod
PYTHONPATH import dir1.dir2.mod
import
import
import import
562 | Chapter 23: Module Packages
import mod.py
www.it-ebooks.info
Package __init__.py Files
import dir0\dir1\dir2\mod.py
import import dir1.dir2.mod
PYTHONPATH
dir0\ dir1\ __init__.py dir2\ __init__.py mod.py
from *
from .. import *
Package Import Basics | 563
www.it-ebooks.info
dir1.dir2
from * __all__ from *
__all__
__all__ from * from *
from submodule import X X __all__
__init__ import
Package Import Example
print('dir1 init') x = 1 print('dir2 init') y = 2
564 | Chapter 23: Module Packages
www.it-ebooks.info
print('in mod.py') z = 3
sys.path import print reload
reload
% python >>> import dir1.dir2.mod dir1 init dir2 init in mod.py >>> >>> import dir1.dir2.mod >>> >>> from imp import reload >>> reload(dir1) dir1 init >>> >>> reload(dir1.dir2) dir2 init
import mod
dir2
dir1 >>> dir1 >>> dir1.dir2 >>> dir1.dir2.mod
dir1.x
x
mod.z
z >>> dir1.x 1 >>> dir1.dir2.y 2 >>> dir1.dir2.mod.z 3
Package Import Example | 565
www.it-ebooks.info
from Versus import with Packages import dir1 z
dir2
mod
>>> dir2.mod NameError: name 'dir2' is not defined >>> mod.z NameError: name 'mod' is not defined
from from import % python >>> from dir1.dir2 import mod dir1 init dir2 init in mod.py >>> mod.z 3 >>> from dir1.dir2.mod import z >>> z 3 >>> import dir1.dir2.mod as mod >>> mod.z 3
Why Use Package Imports?
import utilities
566 | Chapter 23: Module Packages
import as
www.it-ebooks.info
import database.client.utilities
PYTHONPATH
A Tale of Three Systems
import utilities
system1\ utilities.py main.py other.py
import utilities
system2\ utilities.py main.py other.py
Why Use Package Imports? | 567
www.it-ebooks.info
import utilities utilities.func('spam')
sys.path
root\ system1\ __init__.py utilities.py main.py other.py system2\ __init__.py utilities.py main.py other.py system3\ __init__.py myfile.py
import
568 | Chapter 23: Module Packages
www.it-ebooks.info
import system1.utilities import system2.utilities system1.utilities.function('spam') system2.utilities.function('eggs')
import
from
from
import
import utilities
Package Relative Imports import
Package Relative Imports | 569
www.it-ebooks.info
Changes in Python 3.0
from
from
from
Relative Import Basics from
from
sys.path
sys.path
570 | Chapter 23: Module Packages
www.it-ebooks.info
from . import spam
spam from .spam import name
spam name
from __future__ import
absolute_import
sys.path string
sys.path
import string
from __future__
string
from . import string
from import
import modname
from import
Package Relative Imports | 571
www.it-ebooks.info
from .string import name1, name2 from . import string from .. import string
Why Relative Imports?
mypkg\ __init__.py main.py string.py
mypkg main
mypkg.string string string
mypkg.main
mypkg.main
string mypkg.string
string
import spam sys.path string string
The relative imports solution in 3.0 import
572 | Chapter 23: Module Packages
www.it-ebooks.info
string sys.path import string
from from string import name
from from . import string
string import
from .string import name1, name2
string mypkg.main
name1
name2
mypkg.string
from .. import spam
mypkg mypkg
spam A.B.C
from . import D from .. import E from .D import X from ..E import X
Relative imports versus absolute package paths mypkg sys.path from mypkg import string
mypkg Package Relative Imports | 573
www.it-ebooks.info
sys.path from system.section.mypkg import string
from . import string
The Scope of Relative Imports
import
from
import from
sys.path PYTHONPATH
sys.path sys.path
574 | Chapter 23: Module Packages
www.it-ebooks.info
from
from
Module Lookup Rules Summary A sys.path
A.B.C
A.B.C A
sys.path B
A
C
B import
sys.path from
sys.path
from . import A
Relative Imports in Action
Imports outside packages string C:\test> c:\Python30\python >>> import string >>> string
Package Relative Imports | 575
www.it-ebooks.info
print('string' * 8) C:\test> c:\Python30\python >>> import string stringstringstringstringstringstringstringstring >>> string
>>> from . import string Traceback (most recent call last): File "", line 1, in ValueError: Attempted relative import in non-package
sys.path sys.path
import string print(string) C:\test> C:\python30\python main.py stringstringstringstringstringstringstringstring
Imports within packages string
C:\test> del string* C:\test> mkdir pkg import eggs print(eggs.X) X = 99999 import string print(string)
576 | Chapter 23: Module Packages
www.it-ebooks.info
import
C:\test> c:\Python26\python >>> import pkg.spam 99999 C:\test> c:\Python30\python >>> import pkg.spam Traceback (most recent call last): File "", line 1, in File "pkg\spam.py", line 1, in import eggs ImportError: No module named eggs
from . import eggs print(eggs.X) X = 99999 import string print(string) C:\test> c:\Python26\python >>> import pkg.spam 99999 C:\test> c:\Python30\python >>> import pkg.spam 99999
Imports are still relative to the CWD string string
print('string' * 8) from . import eggs
Package Relative Imports | 577
www.it-ebooks.info
print(eggs.X) X = 99999 import string print(string) C:\test> c:\Python30\python >>> import pkg.spam stringstringstringstringstringstringstringstring 99999
Selecting modules with relative and absolute imports string C:\test> del string* import string print(string) print('Ni' * 8)
string
C:\test> c:\Python30\python >>> import pkg.spam C:\test> c:\Python26\python >>> import pkg.spam NiNiNiNiNiNiNiNi
from . import string print(string) print('Ni' * 8) C:\test> c:\Python30\python >>> import pkg.spam NiNiNiNiNiNiNiNi
578 | Chapter 23: Module Packages
www.it-ebooks.info
C:\test> c:\Python26\python >>> import pkg.spam NiNiNiNiNiNiNiNi
from . import string C:\test> C:\python30\python >>> import pkg.spam ...text omitted... ImportError: cannot import name string
Imports are still relative to the CWD (again) sys.path
string
print('string' * 8) from . import string print(string) print('Ni' * 8)
string C:\test> c:\Python30\python >>> import pkg.spam NiNiNiNiNiNiNiNi
print('string' * 8)
Package Relative Imports | 579
www.it-ebooks.info
import string print(string) print('Ni' * 8) C:\test> c:\Python30\python >>> import pkg.spam stringstringstringstringstringstringstringstring C:\test> c:\Python26\python >>> import pkg.spam NiNiNiNiNiNiNiNi
sys.path
580 | Chapter 23: Module Packages
www.it-ebooks.info
Why You Will Care: Module Packages
from win32com.client import constants, Dispatch
client
win32com
from email.message import Message from tkinter.filedialog import askopenfilename from http.server import CGIHTTPRequestHandler
Chapter Summary
from
Chapter Summary | 581
www.it-ebooks.info
__name__
Test Your Knowledge: Quiz
import
from from mypkg import spam
from . import spam
Test Your Knowledge: Answers
from as
import from import
import
import
from
from import
from from mypkg import spam from . import spam sys.path
582 | Chapter 23: Module Packages
mypkg sys.path spam
www.it-ebooks.info
CHAPTER 24
Advanced Module Topics
__future__
__name__
sys.path
__name__
Data Hiding in Modules
583
www.it-ebooks.info
Minimizing from * Damage: _X and __all__ _X from * from *
import _X __all__ __all__ = ["Error", "encode", "decode"]
from * __all__
_X
__all__
_X __all__
from * _X
__all__
from * from *
__all__ from *
Enabling Future Language Features import from __future__ import featurename
featurename
584 | Chapter 24: Advanced Module Topics
generators print
www.it-ebooks.info
Mixed Usage Modes: __name__ and __main__ __name__ __name__ "__main__" __name__ __name__ tester def tester(): print("It's Christmas in Heaven...") if __name__ == '__main__': tester()
% python >>> import runme >>> runme.tester() It's Christmas in Heaven...
% python runme.py It's Christmas in Heaven...
__name__
__name__ __name__
__name__
Mixed Usage Modes: __name__ and __main__ | 585
www.it-ebooks.info
unittest
doctest
__name__
__name__
Unit Tests with __name__ __name__ def minmax(test, *args): res = args[0] for arg in args[1:]: if test(arg, res): res = arg return res def lessthan(x, y): return x < y def grtrthan(x, y): return x > y print(minmax(lessthan, 4, 2, 1, 5, 6, 3)) print(minmax(grtrthan, 4, 2, 1, 5, 6, 3))
__name__
print('I am:', __name__) def minmax(test, *args): res = args[0] for arg in args[1:]: if test(arg, res): res = arg return res def lessthan(x, y): return x < y def grtrthan(x, y): return x > y if __name__ == '__main__': print(minmax(lessthan, 4, 2, 1, 5, 6, 3)) print(minmax(grtrthan, 4, 2, 1, 5, 6, 3))
586 | Chapter 24: Advanced Module Topics
www.it-ebooks.info
__name__ __main__ % python min.py I am: __main__ 1 6
__main__ >>> import min I am: min >>> min.minmax(min.lessthan, 's', 'p', 'a', 'm') 'a'
Using Command-Line Arguments with __name__ __name__
sys.argv
""" Various specialized string display formatting utilities. Test me with canned self-test or command-line arguments. """ def commas(N): """ format positive integer-like N for display with commas between digit groupings: xxx,yyy,zzz """ digits = str(N) assert(digits.isdigit()) result = '' while digits: digits, last3 = digits[:-3], digits[-3:] result = (last3 + ',' + result) if result else last3 return result def money(N, width=0): """
Mixed Usage Modes: __name__ and __main__ | 587
www.it-ebooks.info
format number N for display with commas, 2 decimal digits, leading $ and sign, and optional padding: $ -xxx,yyy.zz """ sign = '-' if N < 0 else '' N = abs(N) whole = commas(int(N)) fract = ('%.2f' % N)[-2:] format = '%s%s.%s' % (sign, whole, fract) return '$%*s' % (width, format) if __name__ == '__main__': def selftest(): tests = 0, 1 # fails: −1, 1.23 tests += 12, 123, 1234, 12345, 123456, 1234567 tests += 2 ** 32, 2 ** 100 for test in tests: print(commas(test)) print('') tests = 0, 1, −1, 1.23, 1., 1.2, 3.14159 tests += 12.34, 12.344, 12.345, 12.346 tests += 2 ** 32, (2 ** 32 + .2345) tests += 1.2345, 1.2, 0.2345 tests += −1.2345, −1.2, −0.2345 tests += −(2 ** 32), −(2**32 + .2345) tests += (2 ** 100), −(2 ** 100) for test in tests: print('%s [%s]' % (money(test, 17), test)) import sys if len(sys.argv) == 1: selftest() else: print(money(float(sys.argv[1]), int(sys.argv[2])))
C:\misc> python formats.py 999999999 0 $999,999,999.00 C:\misc> python formats.py −999999999 0 $-999,999,999.00 C:\misc> python formats.py 123456789012345 0 $123,456,789,012,345.00 C:\misc> python formats.py −123456789012345 25 $ −123,456,789,012,345.00 C:\misc> python formats.py 123.456 0 $123.46
588 | Chapter 24: Advanced Module Topics
www.it-ebooks.info
C:\misc> python formats.py −123.454 0 $-123.45 C:\misc> python formats.py ...canned tests: try this yourself...
>>> from formats import money, commas >>> money(123.456) '$123.46' >>> money(-9999999.99, 15) '$ −9,999,999.99' >>> X = 99999999999999999999 >>> '%s (%s)' % (commas(X), X) '99,999,999,999,999,999,999 (99999999999999999999)'
help >>> import formats >>> help(formats) Help on module formats: NAME
formats
FILE
c:\misc\formats.py
DESCRIPTION Various specialized string display formatting utilities. Test me with canned self-test or command-line arguments. FUNCTIONS commas(N) format positive integer-like N for display with commas between digit groupings: xxx,yyy,zzz money(N, width=0) format number N for display with commas, 2 decimal digits, leading $ and sign, and optional padding: $ -xxx,yyy.zz
getopt
optparse
input
Mixed Usage Modes: __name__ and __main__ | 589
www.it-ebooks.info
{,d}
Changing the Module Search Path PYTHONPATH sys.path sys
path
sys.path
>>> import sys >>> sys.path ['', 'C:\\users', 'C:\\Windows\\system32\\python30.zip', ...more deleted...] >>> sys.path.append('C:\\sourcedir') >>> import string
sys.path >>> sys.path = [r'd:\temp'] >>> sys.path.append('c:\\lp4e\\examples') >>> sys.path ['d:\\temp', 'c:\\lp4e\\examples'] >>> import string Traceback (most recent call last): File "", line 1, in ImportError: No module named string
string sys.path PYTHONPATH
590 | Chapter 24: Advanced Module Topics
www.it-ebooks.info
The as Extension for import and from import
from import
import modulename as name
import modulename name = modulename del modulename
import
as from
from modulename import attrname as name
import import reallylongmodulename as name name.func() from module1 import utility as util1 from module2 import utility as util2 util1(); util2()
import dir1.dir2.mod as mod mod.func()
Modules Are Objects: Metaprograms
name
M __dict__
sys.modules
modules
sys
getattr object.attr attr
Modules Are Objects: Metaprograms | 591
www.it-ebooks.info
M.name M.__dict__['name'] sys.modules['M'].name getattr(M, 'name')
dir listing """ mydir.py: a module that lists the namespaces of other modules """ seplen = 60 sepchr = '-' def listing(module, verbose=True): sepline = sepchr * seplen if verbose: print(sepline) print('name:', module.__name__, 'file:', module.__file__) print(sepline) count = 0 for attr in module.__dict__: print('%02d) %s' % (count, attr), end = ' ') if attr.startswith('__'): print('') else: print(getattr(module, attr)) count += 1 if verbose: print(sepline) print(module.__name__, 'has %d names' % count) print(sepline) if __name__ == '__main__': import mydir listing(mydir)
__doc__
help
sys.modules global X; X=0 import sys; glob=sys.modules[__name__]; glob.X=0
592 | Chapter 24: Advanced Module Topics
global __name__
www.it-ebooks.info
>>> import mydir >>> help(mydir) Help on module mydir: NAME
mydir - mydir.py: a module that lists the namespaces of other modules
FILE
c:\users\veramark\mark\mydir.py
FUNCTIONS listing(module, verbose=True) DATA
sepchr = '-' seplen = 60
print
__future__
end C:\Users\veramark\Mark> c:\Python30\python mydir.py -----------------------------------------------------------name: mydir file: C:\Users\veramark\Mark\mydir.py -----------------------------------------------------------00) seplen 60 01) __builtins__ 02) __file__ 03) __package__ 04) listing 05) __name__ 06) sepchr 07) __doc__ -----------------------------------------------------------mydir has 8 names ------------------------------------------------------------
tkinter Tkinter >>> import mydir >>> import tkinter >>> mydir.listing(tkinter) -----------------------------------------------------------name: tkinter file: c:\PYTHON30\lib\tkinter\__init__.py -----------------------------------------------------------00) getdouble 01) MULTIPLE multiple 02) mainloop 03) Canvas 04) AtSelLast ...many more name omitted... 151) StringVar
Modules Are Objects: Metaprograms | 593
www.it-ebooks.info
152) ARC arc 153) At 154) NSEW nsew 155) SCROLL scroll -----------------------------------------------------------tkinter has 156 names ------------------------------------------------------------
getattr
mydir
Importing Modules by Name String import
from
import >>> import "string" File "", line 1 import "string" ^ SyntaxError: invalid syntax
x = "string" import x
string import
import exec
exec
>>> modname = "string" >>> exec("import " + modname) >>> string
mydir.listing PYTHONSTARTUP __main__
594 | Chapter 24: Advanced Module Topics
www.it-ebooks.info
exec
eval
exec exec
import
__import__ __import__ >>> modname = "string" >>> string = __import__(modname) >>> string
Transitive Module Reloads
A A
B
C
A
B A
C B
B
C
C
import B import C % python >>> . . . >>> from imp import reload >>> reload(A)
reload
reload
A
__dict__
type __dict__
type Transitive Module Reloads | 595
www.it-ebooks.info
reload_all
types
type
visited
visited.add(module) """ reloadall.py: transitively reload nested modules """ import types from imp import reload def status(module): print('reloading ' + module.__name__) def transitive_reload(module, visited): if not module in visited: status(module) reload(module) visited[module] = None for attrobj in module.__dict__.values(): if type(attrobj) == types.ModuleType: transitive_reload(attrobj, visited) def reload_all(*args): visited = {} for arg in args: if type(arg) == types.ModuleType: transitive_reload(arg, visited) if __name__ == '__main__': import reloadall reload_all(reloadall)
reload_all reload
+
print
C:\misc> c:\Python30\python reloadall.py reloading reloadall reloading types
os tkinter
tkinter sys Tkinter tkinter
596 | Chapter 24: Advanced Module Topics
os
www.it-ebooks.info
>>> from reloadall import reload_all >>> import os, tkinter >>> reload_all(os) reloading os reloading copyreg reloading ntpath reloading genericpath reloading stat reloading sys reloading errno >>> reload_all(tkinter) reloading tkinter reloading _tkinter reloading tkinter._fix reloading sys reloading ctypes reloading os reloading copyreg reloading ntpath reloading genericpath reloading stat reloading errno reloading ctypes._endian reloading tkinter.constants
import b X = 1 import c Y = 2 Z = 3 C:\misc> C:\Python30\python >>> import a >>> a.X, a.b.Y, a.b.c.Z (1, 2, 3)
>>> from imp import reload >>> reload(a) >>> a.X, a.b.Y, a.b.c.Z (111, 2, 3) >>> from reloadall import reload_all >>> reload_all(a) reloading a
Transitive Module Reloads | 597
www.it-ebooks.info
reloading b reloading c >>> a.X, a.b.Y, a.b.c.Z (111, 222, 333)
Module Design Concepts
__main__
598 | Chapter 24: Advanced Module Topics
www.it-ebooks.info
Module Gotchas
Statement Order Matters in Top-Level Code
Module Gotchas | 599
www.it-ebooks.info
func1() def func1(): print(func2()) func1() def func2(): return "Hello" func1()
func1 func2
func1
func1 func1
func1 def func2 def func1 func1 func2
def def def
from Copies Names but Doesn’t Link from from
X = 99 def printer(): print(X)
from
from nested1 import X, printer X = 88 printer() % python nested2.py 99
600 | Chapter 24: Advanced Module Topics
www.it-ebooks.info
import
import nested1 nested1.X = 88 nested1.printer() % python nested3.py 88
from * Can Obscure the Meaning of Variables from module import * from * from *
>>> >>> >>> >>>
from module1 import * from module2 import * from module3 import * . . .
>>> func()
from
from * from *
import from
from
reload May Not Impact from Imports from
from from from
Module Gotchas | 601
www.it-ebooks.info
from from from module import X . . . from imp import reload reload(module) X
import
from
import module . . . from imp import reload reload(module) module.X
reload, from, and Interactive Testing from
from from module import function function(1, 2, 3)
from imp import reload reload(module)
from reload
module
function import
from imp import reload import module reload(module) function(1, 2, 3)
reload function module.function reload
from
602 | Chapter 24: Advanced Module Topics
www.it-ebooks.info
from imp import reload import module reload(module) from module import function function(1, 2, 3)
reload
from
from reload reload from reload import exec
Recursive from Imports May Not Work
import from recur1 recur2 recur1
recur2 recur1 Y recur2
X
import from import
X
Y
recur1
X = 1 import recur2 Y = 2 from recur1 import X from recur1 import Y C:\misc> C:\Python30\python >>> import recur1 Traceback (most recent call last): File "", line 1, in
Module Gotchas | 603
www.it-ebooks.info
File "recur1.py", line 2, in import recur2 File "recur2.py", line 2, in from recur1 import Y ImportError: cannot import name Y
recur1 recur2 recur1
recur2 from
import
from
from
Chapter Summary __future__ __name__
Test Your Knowledge: Quiz __name__
604 | Chapter 24: Advanced Module Topics
"__main__"
www.it-ebooks.info
sys.path
PYTHONPATH
__future__
Test Your Knowledge: Answers from * from
import __name__
"__main__"
import
exec
__import__ sys.path PYTHONPATH
Test Your Knowledge: Part V Exercises
countLines(name) file.readlines
len
countChars(name) file.read test(name) input
sys.argv
Test Your Knowledge: Part V Exercises | 605
www.it-ebooks.info
mymod
import PYTHONPATH test("mymod.py") test file.seek(0) from/from *
mymod
from from * mymod
test
__name__
from myclient myclient
"__main__"
mymod myclient
myclient mymod mymod import myclient
import mypkg.mymod
changer fg
606 | Chapter 24: Advanced Module Topics
__dict__
www.it-ebooks.info
recur1 recur2 recur2 sys.modules
recur1
python recur1.py
recur1 recur1 recur2
recur2 recur2
Test Your Knowledge: Part V Exercises | 607
www.it-ebooks.info
www.it-ebooks.info
PART VI
Classes and OOP
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 25
OOP: The Big Picture
class
611
www.it-ebooks.info
Why Use Classes?
612 | Chapter 25: OOP: The Big Picture
www.it-ebooks.info
OOP from 30,000 Feet
Attribute Inheritance Search
object.attribute
class attribute
attribute
object
object.attribute
OOP from 30,000 Feet | 613
www.it-ebooks.info
C1 C2
C3
I1
I2
C2 C1
I2.w
614 | Chapter 25: OOP: The Big Picture
C3
www.it-ebooks.info
object.attribute w
I2 I2, C1, C2, C3
w C3 C3.w w
w I2.w I2
w
C3 w x y
I1.x
I2.x
x
C1
I1.y
I2.y
y
C1
I1.z
I2.z
z
C2
I2.name
name
C1
C1
z
C2 y
C2
C3
I2
x C2 C1
Classes and Instances
Employee Employee
OOP from 30,000 Feet | 615
www.it-ebooks.info
computeSalary hoursWorked
Class Method Calls I2.w C3.w
I2.w I2 C3.w(I2)
C3.w I2.w()
giveRaise Employee
self bob.giveRaise() Employee.giveRaise(bob)
Coding Class Trees class class
class
616 | Chapter 25: OOP: The Big Picture
www.it-ebooks.info
class class C2: ... class C3: ... class C1(C2, C3): ... I1 = C1() I2 = C1()
class C1 C1
class
C1
class def self def
class
def
class C1(C2, C3): def setname(self, who): self.name = who I1 = C1() I2 = C1() I1.setname('bob') I2.setname('mel') print(I1.name)
OOP from 30,000 Feet | 617
www.it-ebooks.info
def def
class self
self self
self self
C1.setname I1.setname C1 setname
C1 name I1.name
I1 I1.setname
name class C1(C2, C3): def __init__(self, who): self.name = who I1 = C1('bob') I2 = C1('mel') print(I1.name)
__init__ self __init__
__init__
self self
618 | Chapter 25: OOP: The Big Picture
this
www.it-ebooks.info
intersect
& __and__
OOP Is About Code Reuse
class Employee: def computeSalary(self): ... def giveRaise(self): ... def promote(self): ... def retire(self): ...
OOP from 30,000 Feet | 619
www.it-ebooks.info
class Engineer(Employee): def computeSalary(self): ...
computeSalary Employee bob = Employee() mel = Engineer()
company = [bob, mel] for emp in company: print(emp.computeSalary())
computeSalary
def processor(reader, converter, writer): while 1: data = reader.read() if not data: break
company shelve
620 | Chapter 25: OOP: The Big Picture
www.it-ebooks.info
data = converter(data) writer.write(data)
read processor class Reader: def read(self): ... def other(self): ... class FileReader(Reader): def read(self): ... class SocketReader(Reader): def read(self): ... ... processor(FileReader(...), Converter, processor(SocketReader(...), Converter, processor(FtpReader(...), Converter,
write
FileWriter(...)) TapeWriter(...)) XmlWriter(...))
read
write
processor converter
Employee Reader
Writer
OOP from 30,000 Feet | 621
www.it-ebooks.info
Chapter Summary
Test Your Knowledge: Quiz
__init__
Test Your Knowledge: Answers
622 | Chapter 25: OOP: The Big Picture
www.it-ebooks.info
self
__init__
__init__
__init__ class
class
Test Your Knowledge: Answers | 623
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 26
Class Coding Basics
class
Classes Generate Multiple Instance Objects
625
www.it-ebooks.info
imp.reload
def
Class Objects Provide Default Behavior class class def
class
class
def
class
class class
def class
class object.name
def
Instance Objects Are Concrete Items
626 | Chapter 26: Class Coding Basics
class
www.it-ebooks.info
self self self
A First Example FirstClass
class
>>> class FirstClass: ... def setdata(self, value): ... self.data = value ... def display(self): ... print(self.data) ...
def class def
setdata
def display class FirstClass.setdata
FirstClass.display
def
>>> x = FirstClass() >>> y = FirstClass()
x FirstClass
y
Classes Generate Multiple Instance Objects | 627
www.it-ebooks.info
>>> x.setdata("King Arthur") >>> y.setdata(3.14159)
x
y
setdata
setdata self.data
FirstClass self x
y data self display
self.data display
x
y
>>> x.display() King Arthur >>> y.display() 3.14159
data
display
setdata data setdata
628 | Chapter 26: Class Coding Basics
www.it-ebooks.info
self >>> x.data = "New value" >>> x.display() New value
>>> x.anothername = "spam"
anothername x self
Classes Are Customized by Inheritance
class class
Classes Are Customized by Inheritance | 629
www.it-ebooks.info
object.attribute class X.attr
self self.attr
attr
self
A Second Example SecondClass
FirstClass
>>> class SecondClass(FirstClass): ... def display(self): ... print('Current value = "%s"' % self.data) ...
SecondClass
display FirstClass SecondClass display
display SecondClass
Class
SecondClass FirstClass
SecondClass display setdata >>> z = SecondClass() >>> z.setdata(42) >>> z.display() Current value = "42"
630 | Chapter 26: Class Coding Basics
First display
FirstClass SecondClass FirstClass
www.it-ebooks.info
SecondClass FirstClass
setdata display
Second
Class
SecondClass FirstClass
FirstClass x
>>> x.display() New value
FirstClass
Classes Are Attributes in Modules class FirstClass class from modulename import FirstClass class SecondClass(FirstClass): def display(self): ...
import modulename class SecondClass(modulename.FirstClass): def display(self): ...
Classes Are Customized by Inheritance | 631
www.it-ebooks.info
class
var = 1 def func(): ... class spam: ... class ham: ... class eggs: ...
class person: ...
import person x = person.person()
person.person person
person from
from person import person x = person()
import person x = person.Person()
632 | Chapter 26: Class Coding Basics
person
www.it-ebooks.info
Classes Can Intercept Python Operators
__X__
__add__ +
__add__
+
* giveRaise
+
promote
Classes Can Intercept Python Operators | 633
www.it-ebooks.info
__init__ __init__
self
A Third Example SecondClass __init__
self
__add__
ThirdClass
ThirdClass
+
__str__ str mul >>> class ThirdClass(SecondClass): ... def __init__(self, value): ... self.data = value ... def __add__(self, other): ... return ThirdClass(self.data + other) ... def __str__(self): ... return '[ThirdClass: %s]' % self.data ... def mul(self, other): ... self.data *= other ... >>> a = ThirdClass('abc') >>> a.display() Current value = "abc" >>> print(a) [ThirdClass: abc] >>> b = a + 'xyz' >>> b.display() Current value = "abcxyz" >>> print(b) [ThirdClass: abcxyz] >>> a.mul(3) >>> print(a) [ThirdClass: abcabcabc]
634 | Chapter 26: Class Coding Basics
www.it-ebooks.info
ThirdClass SecondClass SecondClass
display ThirdClass value
self.data
ThirdClass
__init__ ThirdClass setdata +
print __add__ __add__
self other + self
data
+
print
__str__ __str__
print display
__init__ __add__
__str__ class
__init__ __add__ ThirdClass
mul self
* *
Classes Can Intercept Python Operators | 635
www.it-ebooks.info
Why Use Operator Overloading?
__init__
__init__
The World’s Simplest Python Class class
>>> class rec: pass
pass
class >>> rec.name = 'Bob' >>> rec.age = 40
636 | Chapter 26: Class Coding Basics
www.it-ebooks.info
>>> print(rec.name) Bob
>>> x = rec() >>> y = rec()
>>> x.name, y.name ('Bob', 'Bob')
name
x
name
y
name >>> x.name = 'Sue' >>> rec.name, x.name, y.name ('Bob', 'Sue', 'Bob')
__dict__ __slots__ __X__ >>> rec.__dict__.keys() ['__module__', 'name', 'age', '__dict__', '__weakref__', '__doc__'] >>> list(x.__dict__.keys()) ['name']
The World’s Simplest Python Class | 637
www.it-ebooks.info
>>> list(y.__dict__.keys()) []
name x
name
age
y __class__
>>> x.__class__
__bases__ >>> rec.__bases__ (,)
class
def
class
>>> def upperName(self): ... return self.name.upper()
name >>> upperName(x) 'SUE'
>>> rec.method = upperName >>> x.method() 'SUE' >>> y.method() 'BOB' self
self self
638 | Chapter 26: Class Coding Basics
self
self
www.it-ebooks.info
>>> rec.method(x) 'SUE'
class self
Classes Versus Dictionaries
>>> >>> >>> >>> >>> >>> mel
rec = {} rec['name'] = 'mel' rec['age'] = 45 rec['job'] = 'trainer/writer'
>>> ... >>> >>> >>> >>> >>> 40
class rec: pass
print(rec['name'])
rec.name = 'mel' rec.age = 45 rec.job = 'trainer/writer' print(rec.age)
class class
>>> ... >>> >>> >>>
class rec: pass pers1 = rec() pers1.name = 'mel' pers1.job = 'trainer'
The World’s Simplest Python Class | 639
www.it-ebooks.info
>>> pers1.age = 40 >>> >>> pers2 = rec() >>> pers2.name = 'vls' >>> pers2.job = 'developer' >>> >>> pers1.name, pers2.name ('mel', 'vls')
name age
>>> class Person: ... def __init__(self, name, job): ... self.name = name ... self.job = job ... def info(self): ... return (self.name, self.job) ... >>> rec1 = Person('mel', 'trainer') >>> rec2 = Person('vls', 'developer') >>> >>> rec1.job, rec2.info() ('trainer', ('vls', 'developer'))
name
job
640 | Chapter 26: Class Coding Basics
www.it-ebooks.info
Chapter Summary class
Test Your Knowledge: Quiz
self
Test Your Knowledge: Answers
class
Test Your Knowledge: Answers | 641
www.it-ebooks.info
class class class
class class self class __init__ self self this
__init__
self
642 | Chapter 26: Class Coding Basics
__init__
www.it-ebooks.info
CHAPTER 27
A More Realistic Example
Person Manager
643
www.it-ebooks.info
Step 1: Making Instances Person
self Person
class Person:
Person
Coding Constructors Person
self self
__init__
class Person: def __init__(self, name, job, pay): self.name = name self.job = job self.pay = pay
self self pay
__init__
job self.job
644 | Chapter 27: A More Realistic Example
name job
www.it-ebooks.info
job
self.job
self.job=job
job __init__
job job pay
None None
0
pay
class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay
job
pay
None
0
Person self
self
Testing As You Go
Step 1: Making Instances | 645
www.it-ebooks.info
class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay bob = Person('Bob Smith') sue = Person('Sue Jones', job='dev', pay=100000) print(bob.name, bob.pay) print(sue.name, sue.pay)
bob
job
pay
sue sue
__init__
name
pay
C:\misc> person.py Bob Smith 0 Sue Jones 100000
Person
bob
name
sue
sue bob
pay sue
bob
self
Using Code Two Ways print
646 | Chapter 27: A More Realistic Example
www.it-ebooks.info
__name__
class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay if __name__ == '__main__': bob = Person('Bob Smith') sue = Person('Sue Jones', job='dev', pay=100000) print(bob.name, bob.pay) print(sue.name, sue.pay)
__name__
__main__
C:\misc> person.py Bob Smith 0 Sue Jones 100000 c:\misc> python Python 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) ... >>> import person >>>
Version Portability Note print print c:\misc> c:\python26\python person.py ('Bob Smith', 0) ('Sue Jones', 100000)
Step 1: Making Instances | 647
www.it-ebooks.info
print
print('{0} {1}'.format(bob.name, bob.pay)) print('%s %s' % (bob.name, bob.pay))
Step 2: Adding Behavior Methods
name
>>> name = 'Bob Smith' >>> name.split() ['Bob', 'Smith'] >>> name.split()[-1] 'Smith'
pay
>>> pay = 100000 >>> pay *= 1.10 >>> print(pay) 110000.0
Person bob.name
sue.pay
name
class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay if __name__ == '__main__':
648 | Chapter 27: A More Realistic Example
pay
www.it-ebooks.info
bob = Person('Bob Smith') sue = Person('Sue Jones', job='dev', pay=100000) print(bob.name, bob.pay) print(sue.name, sue.pay) print(bob.name.split()[-1]) sue.pay *= 1.10 print(sue.pay)
bob sue
pay sue append
Bob Smith 0 Sue Jones 100000 Smith 110000.0
Coding Methods
Step 2: Adding Behavior Methods | 649
www.it-ebooks.info
class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay def lastName(self): return self.name.split()[-1] def giveRaise(self, percent): self.pay = int(self.pay * (1 + percent)) if __name__ == '__main__': bob = Person('Bob Smith') sue = Person('Sue Jones', job='dev', pay=100000) print(bob.name, bob.pay) print(sue.name, sue.pay) print(bob.lastName(), sue.lastName()) sue.giveRaise(.10) print(sue.pay)
self lastName bob
self self
lastName giveRaise
self
sue
Bob Smith 0 Sue Jones 100000 Smith Jones 110000
sue int
int
float
Person round(N, 2) decimal %.2f
650 | Chapter 27: A More Realistic Example
{0:.2f}
www.it-ebooks.info
int
money sue
self bob.lastName() bob
self
sue.lastName() sue
self self
giveRaise
bob bob
giveRaise
giveRaise
percent
assert
Step 3: Operator Overloading
bob.name sue.pay
print(sue) sue Bob Smith 0 Sue Jones 100000 Smith Jones
Step 3: Operator Overloading | 651
www.it-ebooks.info
Providing Print Displays
__init__
__str__
__str__
__str__ __init__
__str__
class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay def lastName(self): return self.name.split()[-1] def giveRaise(self, percent): self.pay = int(self.pay * (1 + percent)) def __str__(self): return '[Person: %s, %s]' % (self.name, self.pay) if __name__ == '__main__': bob = Person('Bob Smith') sue = Person('Sue Jones', job='dev', pay=100000) print(bob) print(sue) print(bob.lastName(), sue.lastName()) sue.giveRaise(.10) print(sue)
%
652 | Chapter 27: A More Realistic Example
__str__
www.it-ebooks.info
__str__ [Person: Bob Smith, 0] [Person: Sue Jones, 100000] Smith Jones [Person: Sue Jones, 110000]
__repr__ __str__
__repr__ __str__
__repr__ __str__
Step 4: Customizing Behavior by Subclassing __str__
Coding Subclasses Person Person
Manager
giveRaise
class Manager(Person):
Manager Person
Manager
Step 4: Customizing Behavior by Subclassing | 653
www.it-ebooks.info
Person
Manager Manager Manager
Person Manager
giveRaise Person
class Manager(Person): def giveRaise(self, percent, bonus=.10):
Augmenting Methods: The Bad Way Manager giveRaise
Person
Manager
class Manager(Person): def giveRaise(self, percent, bonus=.10): self.pay = int(self.pay * (1 + percent + bonus))
giveRaise
Manager
Augmenting Methods: The Good Way giveRaise
class Manager(Person): def giveRaise(self, percent, bonus=.10): Person.giveRaise(self, percent + bonus)
654 | Chapter 27: A More Realistic Example
www.it-ebooks.info
self
instance.method(args...)
class.method(instance, args...)
self
giveRaise self
giveRaise Person Manager self.giveRaise() Manager giveRaise Manager self.giveRaise()
Person self Manager.giveRaise
giveRaise Person giveRaise
class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay def lastName(self): return self.name.split()[-1] def giveRaise(self, percent): self.pay = int(self.pay * (1 + percent)) def __str__(self): return '[Person: %s, %s]' % (self.name, self.pay) class Manager(Person):
Step 4: Customizing Behavior by Subclassing | 655
www.it-ebooks.info
def giveRaise(self, percent, bonus=.10): Person.giveRaise(self, percent + bonus) if __name__ == '__main__': bob = Person('Bob Smith') sue = Person('Sue Jones', job='dev', pay=100000) print(bob) print(sue) print(bob.lastName(), sue.lastName()) sue.giveRaise(.10) print(sue) tom = Manager('Tom Jones', 'mgr', 50000) tom.giveRaise(.10) print(tom.lastName()) print(tom)
Manager Manager [Person: Bob [Person: Sue Smith Jones [Person: Sue Jones [Person: Tom
Smith, 0] Jones, 100000] Jones, 110000] Jones, 60000]
bob giveRaise
sue
tom
Manager Person
Manager
lastName Person
Polymorphism in Action if __name__ == '__main__': ... print('--All three--') for object in (bob, sue, tom): object.giveRaise(.10) print(object)
[Person: Bob Smith, [Person: Sue Jones, Smith Jones [Person: Sue Jones, Jones [Person: Tom Jones, --All three--
0] 100000] 110000] 60000]
656 | Chapter 27: A More Realistic Example
__init__
Manager tom __str__
www.it-ebooks.info
[Person: Bob Smith, 0] [Person: Sue Jones, 121000] [Person: Tom Jones, 72000]
object
Person
giveRaise Manager giveRaise
Manager Person
bob
sue
tom
giveRaise sue
tom
giveRaise giveRaise __str__ Person Manager Person
Inherit, Customize, and Extend Manager Person
Manager giveRaise someThingElse
class Person: def lastName(self): ... def giveRaise(self): ... def __str__(self): ... class Manager(Person): def giveRaise(self, ...): ... def someThingElse(self, ...): ... tom = Manager() tom.lastName() tom.giveRaise() tom.someThingElse() print(tom)
Manager
someThingElse Person
Step 4: Customizing Behavior by Subclassing | 657
www.it-ebooks.info
Person
OOP: The Big Idea
giveRaise Manager Person Manager Person Manager
giveRaise Person
Manager
Person giveRaise Person
Manager
Step 5: Customizing Constructors, Too mgr Manager Manager Manager 658 | Chapter 27: A More Realistic Example
www.it-ebooks.info
__init__ giveRaise
Manager
mgr __init__
Person
Manager tom
mgr
class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay def lastName(self): return self.name.split()[-1] def giveRaise(self, percent): self.pay = int(self.pay * (1 + percent)) def __str__(self): return '[Person: %s, %s]' % (self.name, self.pay) class Manager(Person): def __init__(self, name, pay): Person.__init__(self, name, 'mgr', pay) def giveRaise(self, percent, bonus=.10): Person.giveRaise(self, percent + bonus) if __name__ == '__main__': bob = Person('Bob Smith') sue = Person('Sue Jones', job='dev', pay=100000) print(bob) print(sue) print(bob.lastName(), sue.lastName()) sue.giveRaise(.10) print(sue) tom = Manager('Tom Jones', 50000) tom.giveRaise(.10) print(tom.lastName()) print(tom)
__init__ giveRaise self Person
__init__ __init__
Step 5: Customizing Constructors, Too | 659
www.it-ebooks.info
[Person: Bob [Person: Sue Smith Jones [Person: Sue Jones [Person: Tom
Smith, 0] Jones, 100000] Jones, 110000] Jones, 60000]
OOP Is Simpler Than You May Think
self
Other Ways to Combine Classes
660 | Chapter 27: A More Realistic Example
www.it-ebooks.info
Manager Person __getattr__ getattr
giveRaise
Manager
class Person: ...same... class Manager: def __init__(self, name, pay): self.person = Person(name, 'mgr', pay) def giveRaise(self, percent, bonus=.10): self.person.giveRaise(percent + bonus) def __getattr__(self, attr): return getattr(self.person, attr) def __str__(self): return str(self.person) if __name__ == '__main__': ...same...
Manager
Manager
Person __str__ Manager
Manager
Step 5: Customizing Constructors, Too | 661
www.it-ebooks.info
Department
... bob = Person(...) sue = Person(...) tom = Manager(...) class Department: def __init__(self, *args): self.members = list(args) def addMember(self, person): self.members.append(person) def giveRaises(self, percent): for person in self.members: person.giveRaise(percent) def showAll(self): for person in self.members: print(person) development = Department(bob, sue) development.addMember(tom) development.giveRaises(.10) development.showAll()
Department Person Manager
Person
Manager
Catching Built-in Attributes in 3.0 Manager __str__ __str__
__str__
662 | Chapter 27: A More Realistic Example
__getitem__
www.it-ebooks.info
__getattr__
__getattribute__ __str__
Manager Person
__str__ __str__
__getattr__ __getattr__
__getattribute__
Private
Step 6: Using Introspection Tools
tom Manager
Manager
Person Person
__str__ tom __str__
Manager __init__
mgr Person __str__
__str__
Step 6: Using Introspection Tools | 663
www.it-ebooks.info
Special Class Attributes
instance.__class__ __name__ __bases__
object.__dict__
Person >>> from person import Person >>> bob = Person('Bob Smith') >>> print(bob) [Person: Bob Smith, 0] >>> bob.__class__ >>> bob.__class__.__name__ 'Person' >>> list(bob.__dict__.keys()) ['pay', 'job', 'name'] >>> for key in bob.__dict__: print(key, '=>', bob.__dict__[key]) pay => 0 job => None name => Bob Smith >>> for key in bob.__dict__: print(key, '=>', getattr(bob, key)) pay => 0
664 | Chapter 27: A More Realistic Example
from
www.it-ebooks.info
job => None name => Bob Smith
__dict__
__slots__
__dict__
A Generic Display Tool
__str__
__str__
"Assorted class utilities and tools" class AttrDisplay: """ Provides an inheritable print overload method that displays instances with their class names and a name=value pair for each attribute stored on the instance itself (but not attrs inherited from its classes). Can be mixed into any class, and will work on any instance. """ def gatherAttrs(self): attrs = [] for key in sorted(self.__dict__): attrs.append('%s=%s' % (key, getattr(self, key))) return ', '.join(attrs) def __str__(self): return '[%s: %s]' % (self.__class__.__name__, self.gatherAttrs()) if __name__ == '__main__': class TopTest(AttrDisplay): count = 0 def __init__(self): self.attr1 = TopTest.count self.attr2 = TopTest.count+1 TopTest.count += 2
Step 6: Using Introspection Tools | 665
www.it-ebooks.info
class SubTest(TopTest): pass X, Y = TopTest(), SubTest() print(X) print(Y)
help
__str__ C:\misc> classtools.py [TopTest: attr1=0, attr2=1] [SubTest: attr1=2, attr2=3]
Instance Versus Class Attributes classtools self self __dict__ count __class__ __dict__ __bases__ dir __dict__ >>> from person import Person >>> bob = Person('Bob Smith')
>>> bob.__dict__.keys() ['pay', 'job', 'name'] >>> dir(bob) ['__doc__', '__init__', '__module__', '__str__', 'giveRaise', 'job', 'lastName', 'name', 'pay']
666 | Chapter 27: A More Realistic Example
dir
www.it-ebooks.info
>>> list(bob.__dict__.keys()) ['pay', 'job', 'name'] >>> dir(bob) ['__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', ...more lines omitted... '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'giveRaise', 'job', 'lastName', 'name', 'pay']
dict.keys dir
dir
__X__
dir
dir
Name Considerations in Tool Classes AttrDisplay
classtools
__str__
gatherAttrs gather
Attrs gatherAttrs
TopTest
class TopTest(AttrDisplay): .... def gatherAttrs(self): return 'Spam'
__str__ _gatherAttrs _gatherAttrs
Step 6: Using Introspection Tools | 667
www.it-ebooks.info
__gatherAttrs
Our Classes’ Final Form __str__ Person
print Manager Manager AttrDisplay
__str__
from classtools import AttrDisplay class Person(AttrDisplay): """ Create and process person records """ def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay def lastName(self): return self.name.split()[-1] def giveRaise(self, percent): self.pay = int(self.pay * (1 + percent)) class Manager(Person): """ A customized Person with special requirements """ def __init__(self, name, pay): Person.__init__(self, name, 'mgr', pay) def giveRaise(self, percent, bonus=.10): Person.giveRaise(self, percent + bonus) if __name__ == '__main__': bob = Person('Bob Smith') sue = Person('Sue Jones', job='dev', pay=100000) print(bob) print(sue) print(bob.lastName(), sue.lastName()) sue.giveRaise(.10) print(sue) tom = Manager('Tom Jones', 50000) tom.giveRaise(.10)
668 | Chapter 27: A More Realistic Example
Person
www.it-ebooks.info
print(tom.lastName()) print(tom)
#
AttrDisplay
__str__ self tom
Manager
Person Manager
C:\misc> person.py [Person: job=None, name=Bob Smith, pay=0] [Person: job=dev, name=Sue Jones, pay=100000] Smith Jones [Person: job=dev, name=Sue Jones, pay=110000] Jones [Manager: job=mgr, name=Tom Jones, pay=60000]
Step 7 (Final): Storing Objects in a Database
Step 7 (Final): Storing Objects in a Database | 669
www.it-ebooks.info
Pickles and Shelves pickle dbm
anydbm
shelve
pickle
pickle
pickle
pickle shelve shelve pickle
dbm
len in
dict.keys
670 | Chapter 27: A More Realistic Example
shelve pickle
www.it-ebooks.info
Storing Objects on a Shelve Database
from
import person bob = person.Person(...) from person import Person bob = Person(...)
from
shelve
from person import Person, Manager bob = Person('Bob Smith') sue = Person('Sue Jones', job='dev', pay=100000) tom = Manager('Tom Jones', 50000) import shelve db = shelve.open('persondb') for object in (bob, sue, tom): db[object.name] = object db.close()
os time
Step 7 (Final): Storing Objects in a Database | 671
www.it-ebooks.info
C:\misc> makedb.py
Exploring Shelves Interactively open
shelve.open()
shelve bsddb
bsddb
>>> import glob >>> glob.glob('person*') ['person.py', 'person.pyc', 'persondb.bak', 'persondb.dat', 'persondb.dir']
>>> print(open('persondb.dir').read()) 'Tom Jones', (1024, 91) ...more omitted... >>> print(open('persondb.dat', 'rb').read()) b'\x80\x03cperson\nPerson\nq\x00)\x81q\x01}q\x02(X\x03\x00\x00\x00payq\x03K... ...more omitted...
672 | Chapter 27: A More Realistic Example
www.it-ebooks.info
>>> import shelve >>> db = shelve.open('persondb') >>> len(db) 3 >>> list(db.keys()) ['Tom Jones', 'Sue Jones', 'Bob Smith'] >>> bob = db['Bob Smith'] >>> print(bob) [Person: job=None, name=Bob Smith, pay=0] >>> bob.lastName() 'Smith' >>> for key in db: print(key, '=>', db[key]) Tom Jones => [Manager: job=mgr, name=Tom Jones, pay=50000] Sue Jones => [Person: job=dev, name=Sue Jones, pay=100000] Bob Smith => [Person: job=None, name=Bob Smith, pay=0] >>> for key in sorted(db): print(key, '=>', db[key]) Bob Smith => [Person: job=None, name=Bob Smith, pay=0] Sue Jones => [Person: job=dev, name=Sue Jones, pay=100000] Tom Jones => [Manager: job=mgr, name=Tom Jones, pay=50000]
Person
Manager bob lastName
Person self bob bob
sys.path __main__
Step 7 (Final): Storing Objects in a Database | 673
www.it-ebooks.info
Updating Objects on a Shelve
__str__ giveRaise
import shelve db = shelve.open('persondb') for key in sorted(db): print(key, '\t=>', db[key]) sue = db['Sue Jones'] sue.giveRaise(.10) db['Sue Jones'] = sue db.close()
sue
sue c:\misc> updatedb.py Bob Smith => [Person: job=None, name=Bob Smith, pay=0] Sue Jones => [Person: job=dev, name=Sue Jones, pay=100000] Tom Jones => [Manager: job=mgr, name=Tom Jones, pay=50000] c:\misc> updatedb.py Bob Smith => [Person: job=None, name=Bob Smith, pay=0] Sue Jones => [Person: job=dev, name=Sue Jones, pay=110000] Tom Jones => [Manager: job=mgr, name=Tom Jones, pay=50000] c:\misc> updatedb.py Bob Smith => [Person: job=None, name=Bob Smith, pay=0] Sue Jones => [Person: job=dev, name=Sue Jones, pay=121000] Tom Jones => [Manager: job=mgr, name=Tom Jones, pay=50000] c:\misc> updatedb.py
674 | Chapter 27: A More Realistic Example
www.it-ebooks.info
Bob Smith Sue Jones Tom Jones
=> [Person: job=None, name=Bob Smith, pay=0] => [Person: job=dev, name=Sue Jones, pay=133100] => [Manager: job=mgr, name=Tom Jones, pay=50000]
shelve
pickle
c:\misc> python >>> import shelve >>> db = shelve.open('persondb') >>> rec = db['Sue Jones'] >>> print(rec) [Person: job=dev, name=Sue Jones, pay=146410] >>> rec.lastName() 'Jones' >>> rec.pay 146410
pickle
shelve
Future Directions
tkinter Tkinter
Future Directions | 675
www.it-ebooks.info
tkinter
shelve
shelve
676 | Chapter 27: A More Realistic Example
www.it-ebooks.info
Chapter Summary
Test Your Knowledge: Quiz Manager Person giveRaise
Test Your Knowledge: Quiz | 677
www.it-ebooks.info
__dict__
Test Your Knowledge: Answers AttrDisplay
Manager classtools Person
__str__
__str__ Manager AttrDisplay
class pickle
__class__ lastName giveRaise
__str__
678 | Chapter 27: A More Realistic Example
www.it-ebooks.info
__str__ __init__
Manager
print
Person
name address birthday phone email sendmail smptlib AttrDisplay
Test Your Knowledge: Answers | 679
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 28
Class Coding Details
The class Statement class class class
def
class def
class
class
General Form class
681
www.it-ebooks.info
class (superclass,...): data = value def method(self,...): self.member = value
class __init__
Example class class class
class class
class print = if def class
class class
def
>>> class SharedData: ... spam = 42 ... >>> x = SharedData() >>> y = SharedData() >>> x.spam, y.spam (42, 42)
682 | Chapter 28: Class Coding Details
www.it-ebooks.info
spam
class
>>> SharedData.spam = 99 >>> x.spam, y.spam, SharedData.spam (99, 99, 99)
spam >>> x.spam = 88 >>> x.spam, y.spam, SharedData.spam (88, 99, 99)
y.spam x.spam
x
class MixedNames: data = 'spam' def __init__(self, value): self.data = value def display(self): print(self.data, MixedNames.data)
def = class
data data data data self.data
>>> x = MixedNames(1) >>> y = MixedNames(2)
class
__setattr__
The class Statement | 683
www.it-ebooks.info
>>> x.display(); y.display() 1 spam 2 spam
data __init__
self.data data
class
display
self
x.data
self.data Mixed
Names.data
Methods def
instance.method(args...)
class.method(instance, args...)
self
684 | Chapter 28: Class Coding Details
class
www.it-ebooks.info
self this
self self self
Method Example class NextClass: def printer(self, text): self.message = text print(self.message)
printer
class printer
>>> x = NextClass() >>> x.printer('instance call') instance call >>> x.message 'instance call'
printer self x
text printer
'instance call' self self
printer self >>> NextClass.printer(x, 'class call') class call >>> x.message 'class call'
Methods | 685
www.it-ebooks.info
>>> NextClass.printer('bad call') TypeError: unbound method printer() must be called with NextClass instance...
Calling Superclass Constructors __init__ __init__ __init__ class Super: def __init__(self, x): ...default code... class Sub(Super): def __init__(self, x, y): Super.__init__(self, x) ...custom code... I = Sub(1, 2)
Manager
Other Method Call Possibilities
__init__
686 | Chapter 28: Class Coding Details
www.it-ebooks.info
Inheritance class
object.attr
object object
attr
self
Attribute Tree Construction self class class
class
Specializing Inherited Methods
class class __setattr__ __X
Inheritance | 687
www.it-ebooks.info
>>> class Super: ... def method(self): ... print('in Super.method') ... >>> class Sub(Super): ... def method(self): ... print('starting Sub.method') ... Super.method(self) ... print('ending Sub.method') ...
Sub Super Sub
method Sub.method
688 | Chapter 28: Class Coding Details
Super Super.method
www.it-ebooks.info
>>> x = Super() >>> x.method() in Super.method >>> x = Sub() >>> x.method() starting Sub.method in Super.method ending Sub.method
Class Interface Techniques Super method
delegate
action
Inheritor Super Replacer Super
method
Extender Super
method
Provider action
Super
delegate
class Super: def method(self): print('in Super.method') def delegate(self): self.action() class Inheritor(Super): pass class Replacer(Super): def method(self): print('in Replacer.method') class Extender(Super): def method(self): print('starting Extender.method') Super.method(self) print('ending Extender.method')
Inheritance | 689
www.it-ebooks.info
class Provider(Super): def action(self): print('in Provider.action') if __name__ == '__main__': for klass in (Inheritor, Replacer, Extender): print('\n' + klass.__name__ + '...') klass().method() print('\nProvider...') x = Provider() x.delegate()
for __name__
% python specialize.py Inheritor... in Super.method Replacer... in Replacer.method Extender... starting Extender.method in Super.method ending Extender.method Provider... in Provider.action
Abstract Superclasses Provider delegate
Provider x.delegate Provider
delegate
Super x
self Super.delegate self action delegate
690 | Chapter 28: Class Coding Details
self.action self Provider
Provider
www.it-ebooks.info
NotImplementedError
assert raise
assert class Super: def delegate(self): self.action() def action(self): assert False, 'action must be defined!' >>> X = Super() >>> X.delegate() AssertionError: action must be defined!
assert
NotImplementedError class Super: def delegate(self): self.action() def action(self): raise NotImplementedError('action must be defined!') >>> X = Super() >>> X.delegate() NotImplementedError: action must be defined!
>>> class Sub(Super): pass ... >>> X = Sub() >>> X.delegate() NotImplementedError: action must be defined! >>> class Sub(Super): ... def action(self): print('spam') ... >>> X = Sub() >>> X.delegate() spam
Inheritance | 691
www.it-ebooks.info
Python 2.6 and 3.0 Abstract Superclasses
class from abc import ABCMeta, abstractmethod class Super(metaclass=ABCMeta): @abstractmethod def method(self, ...): pass
class Super: __metaclass__ = ABCMeta @abstractmethod def method(self, ...): pass
>>> from abc import ABCMeta, abstractmethod >>> >>> class Super(metaclass=ABCMeta): ... def delegate(self): ... self.action() ... @abstractmethod ... def action(self): ... pass ... >>> X = Super() TypeError: Can't instantiate abstract class Super with abstract methods action >>> class Sub(Super): pass ... >>> X = Sub() TypeError: Can't instantiate abstract class Sub with abstract methods action >>> class Sub(Super): ... def action(self): print('spam') ... >>> X = Sub() >>> X.delegate() spam
692 | Chapter 28: Class Coding Details
@
www.it-ebooks.info
Namespaces: The Whole Story
X object.X
Simple Names: Global Unless Assigned X = value X X X
Attribute Names: Object Namespaces
Namespaces: The Whole Story | 693
www.it-ebooks.info
object.X = value X
object
object.X X X
object
object
The “Zen” of Python Namespaces: Assignments Classify Names
X = 11 def f(): print(X) def g(): X = 22 print(X) class C: X = 33 def m(self): X = 44 self.X = 55
X X X 22 55
11 33
44 X
X
694 | Chapter 28: Class Coding Details
www.it-ebooks.info
if __name__ == '__main__': print(X) f() g() print(X) obj = C() print(obj.X) obj.m() print(obj.X) print(C.X) #print(C.m.X) #print(g.X)
X C.X def def
import manynames X = 66 print(X) print(manynames.X) manynames.f() manynames.g() print(manynames.C.X) I = manynames.C() print(I.X) I.m() print(I.X)
manynames.f()
X
manynames
X
X
I.m() __init__
Namespaces: The Whole Story | 695
www.it-ebooks.info
global
nonlocal
X = 11 def g1(): print(X) def g2(): global X X = 22 def h1(): X = 33 def nested(): print(X) def h2(): X = 33 def nested(): nonlocal X X = 44
Namespace Dictionaries __dict__
>>> class super: ... def hello(self): ... self.data1 = 'spam' ... >>> class sub(super): ... def hola(self):
696 | Chapter 28: Class Coding Details
www.it-ebooks.info
self.data2 = 'eggs'
... ...
__class__ __bases__
>>> X = sub() >>> X.__dict__ {} >>> X.__class__ >>> sub.__bases__ (,) >>> super.__bases__ (,)
self
self >>> Y = sub() >>> X.hello() >>> X.__dict__ {'data1': 'spam'} >>> X.hola() >>> X.__dict__ {'data1': 'spam', 'data2': 'eggs'} >>> sub.__dict__.keys() ['__module__', '__doc__', 'hola'] >>> super.__dict__.keys() ['__dict__', '__module__', '__weakref__', 'hello', '__doc__'] >>> Y.__dict__ {}
__doc__
Namespaces: The Whole Story | 697
www.it-ebooks.info
Y X
>>> X.data1, X.__dict__['data1'] ('spam', 'spam') >>> X.data3 = 'toast' >>> X.__dict__ {'data1': 'spam', 'data3': 'toast', 'data2': 'eggs'} >>> X.__dict__['data3'] = 'ham' >>> X.data3 'ham'
X.hello
X.__dict__['hello'] dir dir(object) object.__dict__.keys()
dir dir object
>>> X.__dict__, Y.__dict__ ({'data1': 'spam', 'data3': 'ham', 'data2': 'eggs'}, {}) >>> list(X.__dict__.keys()) ['data1', 'data3', 'data2']
>>>> dir(X) ['__doc__', '__module__', 'data1', 'data2', 'data3', 'hello', 'hola'] >>> dir(sub) ['__doc__', '__module__', 'hello', 'hola'] >>> dir(super) ['__doc__', '__module__', 'hello']
dir dir dir
698 | Chapter 28: Class Coding Details
www.it-ebooks.info
>>> dir(X) ['__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', ...more omitted... 'data1', 'data2', 'data3', 'hello', 'hola'] >>> dir(sub) ['__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', ...more omitted... 'hello', 'hola'] >>> dir(super) ['__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', ...more omitted... 'hello' ]
Namespace Links __class__
__bases__
""" Climb inheritance trees using namespace links, displaying higher superclasses with indentation """ def classtree(cls, indent): print('.' * indent + cls.__name__) for supercls in cls.__bases__: classtree(supercls, indent+3) def instancetree(inst): print('Tree of %s' % inst) classtree(inst.__class__, 3) def selftest(): class A: class B(A): class C(A): class D(B,C): class E: class F(D,E):
pass pass pass pass pass pass
Namespaces: The Whole Story | 699
www.it-ebooks.info
instancetree(B()) instancetree(F()) if __name__ == '__main__': selftest()
classtree __name__ __bases__ cls
indent
classtree
C:\misc> c:\python26\python classtree.py Tree of ...B ......A Tree of ...F ......D .........B ............A .........C ............A ......E
object
C:\misc> c:\python30\python classtree.py Tree of ...B ......A .........object Tree of ...F ......D .........B ............A ...............object .........C ............A ...............object ......E .........object
700 | Chapter 28: Class Coding Details
www.it-ebooks.info
C:\misc> c:\python30\python >>> class Emp: pass ... >>> class Person(Emp): pass >>> bob = Person() >>> import classtree >>> classtree.instancetree(bob) Tree of ...Person ......Emp .........object
Documentation Strings Revisited __doc__ def
"I am: docstr.__doc__" def func(args): "I am: docstr.func.__doc__" pass class spam: "I am: spam.__doc__ or docstr.spam.__doc__" def method(self, arg): "I am: spam.method.__doc__ or self.method.__doc__" pass
Documentation Strings Revisited | 701
www.it-ebooks.info
__doc__ >>> import docstr >>> docstr.__doc__ 'I am: docstr.__doc__' >>> docstr.func.__doc__ 'I am: docstr.func.__doc__' >>> docstr.spam.__doc__ 'I am: spam.__doc__ or docstr.spam.__doc__' >>> docstr.spam.method.__doc__ 'I am: spam.method.__doc__ or self.method.__doc__'
object
>>> help(docstr) Help on module docstr: NAME
docstr - I am: docstr.__doc__
FILE
c:\misc\docstr.py
CLASSES spam class spam | I am: spam.__doc__ or docstr.spam.__doc__ | | Methods defined here: | | method(self, arg) | I am: spam.method.__doc__ or self.method.__doc__ FUNCTIONS func(args) I am: docstr.func.__doc__
#
702 | Chapter 28: Class Coding Details
www.it-ebooks.info
Classes Versus Modules
class
Chapter Summary
Test Your Knowledge: Quiz class
Test Your Knowledge: Quiz | 703
www.it-ebooks.info
__init__
Test Your Knowledge: Answers
X = Y
class Class.X
__init__ __init__
self
Superclass.__init__(self, ...)
self Superclass.method(self, ...)
704 | Chapter 28: Class Coding Details
www.it-ebooks.info
CHAPTER 29
Operator Overloading
The Basics
705
www.it-ebooks.info
Constructors and Expressions: __init__ and __sub__ Number __init__ __sub__ class Number: def __init__(self, start): self.data = start def __sub__(self, other): return Number(self.data - other) >>> >>> >>> >>> 3
from number import Number X = Number(5) Y = X – 2 Y.data
__init__
Common Operator Overloading Methods
__add__ __radd__
__iadd__
Method
Implements
Called for
__init__
Constructor
Object creation: X = Class(args)
__del__
Destructor
Object reclamation of X
__add__
Operator +
X + Y, X += Y if no __iadd__
__or__
Operator | (bitwise OR)
X | Y, X |= Y if no __ior__
__repr__, __str__
Printing, conversions
print(X), repr(X), str(X)
__call__
Function calls
X(*args, **kargs)
__getattr__
Attribute fetch
X.undefined
__setattr__
Attribute assignment
X.any = value
__delattr__
Attribute deletion
del X.any
__getattribute__
Attribute fetch
X.any
706 | Chapter 29: Operator Overloading
www.it-ebooks.info
Method
Implements
Called for
__getitem__
Indexing, slicing, iteration
X[key], X[i:j], for loops and other iterations if no __iter__
__setitem__
Index and slice assignment
X[key] = value, X[i:j] = sequence
__delitem__
Index and slice deletion
del X[key], del X[i:j]
__len__
Length
len(X), truth tests if no __bool__
__bool__
Boolean tests
bool(X), truth tests (named __nonzero__ in 2.6)
__lt__, __gt__, __le__, __ge__, __eq__, __ne__
Comparisons
X < Y, X > Y, X = Y, X == Y, X != Y (or else __cmp__ in 2.6 only)
__radd__
Right-side operators
Other + X
__iadd__
In-place augmented operators
X += Y (or else __add__)
__iter__, __next__
Iteration contexts
I=iter(X), next(I); for loops, in if no __contains__, all comprehensions, map(F,X), others (__next__ is named next in 2.6)
__contains__
Membership test
item in X (any iterable)
__index__
Integer value
hex(X), bin(X), oct(X), O[X], O[X:] (replaces Python 2 __oct__, __hex__)
__enter__, __exit__
Context manager (Chapter 33)
with obj as var:
__get__, __set__, __delete__
Descriptor attributes (Chapter 37)
X.attr, X.attr = value, del X.attr
__new__
Creation (Chapter 39)
Object creation, before __init__
__add__ __add__
+
object
__init__ __init__
The Basics | 707
www.it-ebooks.info
Indexing and Slicing: __getitem__ and __setitem__ __getitem__ X X[i]
__getitem__
X
>>> class Indexer: ... def __getitem__(self, index): ... return index ** 2 ... >>> X = Indexer() >>> X[2] 4 >>> for i in range(5): ... print(X[i], end=' ') ... 0 1 4 9 16
Intercepting Slices __getitem__
>>> >>> [7, >>> [6, >>> [5, >>> [5,
L = [5, 6, 7, 8, 9] L[2:4] 8] L[1:] 7, 8, 9] L[:-1] 6, 7, 8] L[::2] 7, 9]
>>> [7, >>> [6, >>> [5, >>> [5,
L[slice(2, 4)] 8] L[slice(1, None)] 7, 8, 9] L[slice(None, −1)] 6, 7, 8] L[slice(None, None, 2)] 7, 9]
708 | Chapter 29: Operator Overloading
www.it-ebooks.info
__getitem__
>>> class Indexer: ... data = [5, 6, 7, 8, 9] ... def __getitem__(self, index): ... print('getitem:', index) ... return self.data[index] ... >>> X = Indexer() >>> X[0] getitem: 0 5 >>> X[1] getitem: 1 6 >>> X[-1] getitem: −1 9
>>> X[2:4] getitem: slice(2, 4, None) [7, 8] >>> X[1:] getitem: slice(1, None, None) [6, 7, 8, 9] >>> X[:-1] getitem: slice(None, −1, None) [5, 6, 7, 8] >>> X[::2] getitem: slice(None, None, 2) [5, 7, 9]
__setitem__
def __setitem__(self, index, value): ... self.data[index] = value
__getitem__
Slicing and Indexing in Python 2.6 __getslice__
__setslice__
__getitem__
__setitem__
Indexing and Slicing: __getitem__ and __setitem__ | 709
www.it-ebooks.info
__getitem__
__setitem__
__index__
>>> class C: ... def __index__(self): ... return 255 ... >>> X = C() >>> hex(X) '0xff' >>> bin(X) '0b11111111' >>> oct(X) '0o377'
__getitem__ >>> ('C' * 256)[255] 'C' >>> ('C' * 256)[X] 'C' >>> ('C' * 256)[X:] 'C'
hex
oct
__hex__
__oct__
Index Iteration: __getitem__ for __getitem__ for
__getitem__
>>> class stepper: ... def __getitem__(self, i): ... return self.data[i] ...
710 | Chapter 29: Operator Overloading
www.it-ebooks.info
>>> >>> >>> >>> 'p' >>> ... ... S p
X = stepper() X.data = "Spam" X[1] for item in X: print(item, end=' ') a m
for
in
map __getitem__ >>> 'p' in X True >>> [c for c in X] ['S', 'p', 'a', 'm'] >>> list(map(str.upper, X)) ['S', 'P', 'A', 'M'] >>> (a, b, c, d) = X >>> a, c, d ('S', 'a', 'm') >>> list(X), tuple(X), ''.join(X) (['S', 'p', 'a', 'm'], ('S', 'p', 'a', 'm'), 'Spam') >>> X
Iterator Objects: __iter__ and __next__ __getitem__ __iter__ __getitem__
__iter__
__getitem__ iter
__iter__ __next__
Iterator Objects: __iter__ and __next__ | 711
www.it-ebooks.info
StopIteration __getitem__ IndexError
__iter__ next next(I)
I.__next__()
I.__next__() next(I) I.next() I.__next__()
I.next()
User-Defined Iterators __iter__
class Squares: def __init__(self, start, stop): self.value = start - 1 self.stop = stop def __iter__(self): return self def __next__(self): if self.value == self.stop: raise StopIteration self.value += 1 return self.value ** 2 % python >>> from iters import Squares >>> for i in Squares(1, 5): ... print(i, end=' ') ... 1 4 9 16 25
self
raise >>> >>> >>> 1 >>> 4
X = Squares(1, 5) I = iter(X) next(I) next(I)
712 | Chapter 29: Operator Overloading
__next__
www.it-ebooks.info
...more omitted... >>> next(I) 25 >>> next(I) StopIteration
__getitem__ for 0..N start..stop
__iter__
next
__getitem__ __iter__ __getitem__
>>> X = Squares(1, 5) >>> X[1] AttributeError: Squares instance has no attribute '__getitem__'
__iter__ __getitem__ __getitem__ __iter__ Squares >>> >>> [1, >>> [] >>> [1, >>> [1,
X = Squares(1, 5) [n for n in X] 4, 9, 16, 25] [n for n in X] [n for n in Squares(1, 5)] 4, 9, 16, 25] list(Squares(1, 3)) 4, 9]
>>> def gsquares(start, stop): ... for i in range(start, stop+1): ... yield i ** 2 ... >>> for i in gsquares(1, 5): ... print(i, end=' ') ... 1 4 9 16 25
for
map
Iterator Objects: __iter__ and __next__ | 713
www.it-ebooks.info
>>> [x ** 2 for x in range(1, 6)] [1, 4, 9, 16, 25]
Multiple Iterators on One Object
>>> S = 'ace' >>> for x in S: ... for y in S: ... print(x + y, end=' ') ... aa ac ae ca cc ce ea ec ee
iter
map range
__iter__ self
class SkipIterator: def __init__(self, wrapped): self.wrapped = wrapped self.offset = 0 def __next__(self): if self.offset >= len(self.wrapped): raise StopIteration else: item = self.wrapped[self.offset] self.offset += 2 return item class SkipObject:
714 | Chapter 29: Operator Overloading
zip
www.it-ebooks.info
def __init__(self, wrapped): self.wrapped = wrapped def __iter__(self): return SkipIterator(self.wrapped) if __name__ == '__main__': alpha = 'abcdef' skipper = SkipObject(alpha) I = iter(skipper) print(next(I), next(I), next(I)) for x in skipper: for y in skipper: print(x + y, end=' ')
% python skipper.py a c e aa ac ae ca cc ce ea ec ee
Squares Squares SkipObject
>>> S = 'abcdef' >>> for x in S[::2]: ... for y in S[::2]: ... print(x + y, end=' ') ... aa ac ae ca cc ce ea ec ee
>>> S = 'abcdef' >>> S = S[::2] >>> S 'ace' >>> for x in S: ... for y in S: ... print(x + y, end=' ') ... aa ac ae ca cc ce ea ec ee
Iterator Objects: __iter__ and __next__ | 715
www.it-ebooks.info
Membership: __contains__, __iter__, and __getitem__
__lt__ __cmp__ __cmp__ __bool__
True False __len__
True __nonzero__
__bool__ in __getitem__ __contains__
__iter__ __iter__ __contains__
class Iters: def __init__(self, value): self.data = value def __getitem__(self, i): print('get[%s]:' % i, end='') return self.data[i] def __iter__(self): print('iter=> ', end='') self.ix = 0 return self def __next__(self): print('next:', end='')
716 | Chapter 29: Operator Overloading
__getitem__
www.it-ebooks.info
if self.ix == len(self.data): raise StopIteration item = self.data[self.ix] self.ix += 1 return item def __contains__(self, x): print('contains: ', end='') return x in self.data X = Iters([1, 2, 3, 4, 5]) print(3 in X) for i in X: print(i, end=' | ') print() print([i ** 2 for i in X]) print( list(map(bin, X)) ) I = iter(X) while True: try: print(next(I), end=' @ ') except StopIteration: break
__contains__ __iter__ __getitem__
__next__
contains: True iter=> next:1 | next:2 | next:3 | next:4 | next:5 | next: iter=> next:next:next:next:next:next:[1, 4, 9, 16, 25] iter=> next:next:next:next:next:next:['0b1', '0b10', '0b11', '0b100', '0b101'] iter=> next:1 @ next:2 @ next:3 @ next:4 @ next:5 @ next:
__contains__ __iter__ iter=> iter=> iter=> iter=> iter=>
next:next:next:True next:1 | next:2 | next:3 | next:4 | next:5 | next: next:next:next:next:next:next:[1, 4, 9, 16, 25] next:next:next:next:next:next:['0b1', '0b10', '0b11', '0b100', '0b101'] next:1 @ next:2 @ next:3 @ next:4 @ next:5 @ next:
__contains__
__iter__
__getitem__ get[0]:get[1]:get[2]:True get[0]:1 | get[1]:2 | get[2]:3 | get[3]:4 | get[4]:5 | get[5]: get[0]:get[1]:get[2]:get[3]:get[4]:get[5]:[1, 4, 9, 16, 25] get[0]:get[1]:get[2]:get[3]:get[4]:get[5]:['0b1', '0b10', '0b11', '0b100','0b101'] get[0]:1 @ get[1]:2 @ get[2]:3 @ get[3]:4 @ get[4]:5 @ get[5]:
Membership: __contains__, __iter__, and __getitem__ | 717
www.it-ebooks.info
__getitem__ __getitem__
>>> X = Iters('spam') >>> X[0] get[0]:'s' >>> 'spam'[1:] 'pam' >>> 'spam'[slice(1, None)] 'pam' >>> X[1:] get[slice(1, None, None)]:'pam' >>> X[:-1] get[slice(None, −1, None)]:'spa'
__iter__ __contains__
Attribute Reference: __getattr__ and __setattr__ __getattr__
__getattr__ >>> class empty: ... def __getattr__(self, attrname): ... if attrname == "age": ... return 40 ... else: ... raise AttributeError, attrname ... >>> X = empty() >>> X.age 40 >>> X.name ...error text omitted... AttributeError: name
empty X.age attrname
X __getattr__
self
X "age"
age X.age
40
age
718 | Chapter 29: Operator Overloading
www.it-ebooks.info
__getattr__ AttributeError X.name
__getattr__
__setattr__ self.attr = value
self.__setattr__('attr', value) self __setattr__
__setattr__
self.__dict__['name'] = x
self.name = x
>>> class accesscontrol: ... def __setattr__(self, attr, value): ... if attr == 'age': ... self.__dict__[attr] = value ... else: ... raise AttributeError, attr + ' not allowed' ... >>> X = accesscontrol() >>> X.age = 40 >>> X.age 40 >>> X.name = 'mel' ...text omitted... AttributeError: name not allowed
Other Attribute Management Tools __getattribute__ __getattr__ property __get__
__set__
Attribute Reference: __getattr__ and __setattr__ | 719
www.it-ebooks.info
Emulating Privacy for Instance Attributes: Part 1 class PrivateExc(Exception): pass class Privacy: def __setattr__(self, attrname, value): if attrname in self.privates: raise PrivateExc(attrname, self) else: self.__dict__[attrname] = value class Test1(Privacy): privates = ['age'] class Test2(Privacy): privates = ['name', 'pay'] def __init__(self): self.__dict__['name'] = 'Tom' x = Test1() y = Test2() x.name = 'Bob' y.name = 'Sue' y.age x.age
= 30 = 40
__getattr__
720 | Chapter 29: Operator Overloading
www.it-ebooks.info
String Representation: __repr__ and __str__ __init__
self.data
__add__ __repr__
__repr__
__str__
>>> class adder: ... def __init__(self, value=0): ... self.data = value ... def __add__(self, other): ... self.data += other ... >>> x = adder() >>> print(x) >>> x
>>> class addrepr(adder): ... def __repr__(self): ... return 'addrepr(%s)' % self.data ... >>> x = addrepr(2) >>> x + 1 >>> x addrepr(3) >>> print(x) addrepr(3) >>> str(x), repr(x) ('addrepr(3)', 'addrepr(3)')
__str__
print print
str
__repr__
repr print
__repr__
str
__str__
print
str __repr__
__str__ __str__
String Representation: __repr__ and __str__ | 721
www.it-ebooks.info
__repr__
__str__
>>> class addstr(adder): ... def __str__(self): ... return '[Value: %s]' % self.data ... >>> x = addstr(3) >>> x + 1 >>> x >>> print(x) [Value: 4] >>> str(x), repr(x) ('[Value: 4]', '')
__repr__ __str__ __repr__
__str__
__repr__ >>> class addboth(adder): ... def __str__(self): ... return '[Value: %s]' % self.data ... def __repr__(self): ... return 'addboth(%s)' % self.data ... >>> x = addboth(4) >>> x + 1 >>> x addboth(5) >>> print(x) [Value: 5] >>> str(x), repr(x) ('[Value: 5]', 'addboth(5)')
__str__ __repr__ __str__ __repr__ >>> class Printer: ... def __init__(self, val): ... self.val = val ... def __str__(self): ... return str(self.val) ... >>> objs = [Printer(2), Printer(3)] >>> for x in objs: print(x) ...
722 | Chapter 29: Operator Overloading
www.it-ebooks.info
2 3 >>> print(objs) [, >> objs [, >> ... ... ... ... ... >>> >>> ... 2 3 >>> [2, >>> [2,
__str__
class Printer: def __init__(self, val): self.val = val def __repr__(self): return str(self.val) objs = [Printer(2), Printer(3)] for x in objs: print(x)
print(objs) 3] objs 3]
__str__
__repr__ __init__
Right-Side and In-Place Addition: __radd__ and __iadd__ __add__ + __radd__ +
__radd__ __add__ >>> class Commuter: ... def __init__(self, val): ... self.val = val ... def __add__(self, other): ... print('add', self.val, other) ... return self.val + other ... def __radd__(self, other): ... print('radd', self.val, other) ... return other + self.val ... >>> x = Commuter(88) >>> y = Commuter(99)
Right-Side and In-Place Addition: __radd__ and __iadd__ | 723
www.it-ebooks.info
>>> x + 1 add 88 1 89 >>> 1 + y radd 99 1 100 >>> x + y add 88 radd 99 88 187
other
x
,
__radd__ self y
+
__add__ __radd__
__add__
Commuter val __radd__
isinstance Commuter
>>> class Commuter: ... def __init__(self, val): ... self.val = val ... def __add__(self, other): ... if isinstance(other, Commuter): other = other.val ... return Commuter(self.val + other) ... def __radd__(self, other): ... return Commuter(other + self.val) ... def __str__(self): ... return '' % self.val ... >>> x = Commuter(88) >>> y = Commuter(99) >>> print(x + 10) >>> print(10 + y) >>> z = x + y >>> print(z) >>> print(z + 10) >>> print(z + z)
724 | Chapter 29: Operator Overloading
www.it-ebooks.info
In-Place Addition +=
__iadd__ Commuter
__add__ += >>> ... ... ... ... ... ... >>> >>> >>> >>> 7 >>> ... ... ... ... ... >>> >>> >>> >>> 7
__iadd__
class Number: def __init__(self, val): self.val = val def __iadd__(self, other): self.val += other return self x = Number(5) x += 1 x += 1 x.val class Number: def __init__(self, val): self.val = val def __add__(self, other): return Number(self.val + other) x = Number(5) x += 1 x += 1 x.val
__mul__ __rmul__
__imul__
Vector
Employee
Button
Call Expressions: __call__ __call__ __call__
>>> class Callee: ... def __call__(self, *pargs, **kargs): ... print('Called:', pargs, kargs) ... >>> C = Callee() >>> C(1, 2, 3)
Call Expressions: __call__ | 725
www.it-ebooks.info
Called: (1, 2, 3) {} >>> C(1, 2, 3, x=4, y=5) Called: (1, 2, 3) {'y': 5, 'x': 4}
__call__
class C: def __call__(self, a, b, c=5, d=6): ... class C: def __call__(self, *pargs, **kargs): ... class C: def __call__(self, *pargs, d=6, **kargs): ...
X = C() X(1, 2) X(1, 2, 3, 4) X(a=1, b=2, d=4) X(*[1, 2], **dict(c=3, d=4)) X(1, *(2,), c=3, **dict(d=4))
__call__
>>> ... ... ... ... ... >>> >>> 6 >>> 8
class Prod: def __init__(self, value): self.value = value def __call__(self, other): return self.value * other x = Prod(2) x(3) x(4)
__call__ >>> class Prod: ... def __init__(self, value): ... self.value = value ... def comp(self, other): ... return self.value * other ...
726 | Chapter 29: Operator Overloading
www.it-ebooks.info
>>> x = Prod(3) >>> x.comp(3) 9 >>> x.comp(4) 12
__call__
__init__
__str__
__repr__
Function Interfaces and Callback-Based Code tkinter
Tkinter tkinter
__call__
x.comp
x bound __call__
class Callback: def __init__(self, color): self.color = color def __call__(self): print('turn', self.color)
cb1 = Callback('blue') cb2 = Callback('green') B1 = Button(command=cb1) B2 = Button(command=cb2)
cb1() cb2()
Call Expressions: __call__ | 727
www.it-ebooks.info
lambda cb3 = (lambda color='red': 'turn ' + color) print(cb3())
self class Callback: def __init__(self, color): self.color = color def changeColor(self): print('turn', self.color) cb1 = Callback('blue') cb2 = Callback('yellow') B1 = Button(command=cb1.changeColor) B2 = Button(command=cb2.changeColor)
changeColor object = Callback('blue') cb = object.changeColor cb()
__call__ __call__ __call__
Comparisons: __lt__, __gt__, and Others < > = ==
!=
728 | Chapter 29: Operator Overloading
www.it-ebooks.info
__add__ __radd__ __lt__
==
!=
__gt__
__eq__
__ne__
__cmp__
self cmp(x, y)
__cmp__
cmp
class C: data = 'spam' def __gt__(self, other): return self.data > other def __lt__(self, other): return self.data < other X = C() print(X > 'ham') print(X < 'ham')
The 2.6 __cmp__ Method (Removed in 3.0) __cmp__ __cmp__ class C: data = 'spam' def __cmp__(self, other): return cmp(self.data, other) X = C() print(X > 'ham') print(X < 'ham')
Comparisons: __lt__, __gt__, and Others | 729
www.it-ebooks.info
__cmp__
cmp
cmp class C: data = 'spam' def __cmp__(self, other): return (self.data > other) - (self.data < other)
__cmp__ __cmp__ __getslice__ __cmp__
Boolean Tests: __bool__ and __len__ __bool__ __len__
>>> class Truth: ... def __bool__(self): return True ... >>> X = Truth() >>> if X: print('yes!') ... yes! >>> class Truth: ... def __bool__(self): return False ... >>> X = Truth() >>> bool(X) False
>>> class Truth: ... def __len__(self): return 0 ... >>> X = Truth() >>> if not X: print('no!')
730 | Chapter 29: Operator Overloading
www.it-ebooks.info
... no!
__bool__
__len__
>>> class Truth: ... def __bool__(self): return True ... def __len__(self): return 0 ... >>> X = Truth() >>> if X: print('yes!') ... yes!
>>> class Truth: ... pass ... >>> X = Truth() >>> bool(X) True
Booleans in Python 2.6 __nonzero__ __nonzero__
__bool__
__bool__ __len__ __bool__ False
C:\misc> c:\python30\python >>> class C: ... def __bool__(self): ... print('in bool') ... return False ... >>> X = C() >>> bool(X) in bool False >>> if X: print(99) ... in bool
Boolean Tests: __bool__ and __len__ | 731
www.it-ebooks.info
__bool__ C:\misc> c:\python26\python >>> class C: ... def __bool__(self): ... print('in bool') ... return False ... >>> X = C() >>> bool(X) True >>> if X: print(99) ... 99
__nonzero__
0
__len__
C:\misc> c:\python26\python >>> class C: ... def __nonzero__(self): ... print('in nonzero') ... return False ... >>> X = C() >>> bool(X) in nonzero False >>> if X: print(99) ... in nonzero
__nonzero__ __bool__
Object Destruction: __del__ __init__ __del__ >>> class Life: ... def __init__(self, name='unknown'): ... print('Hello', name) ... self.name = name ... def __del__(self): ... print('Goodbye', self.name) ... >>> brian = Life('Brian') Hello Brian >>> brian = 'loretta' Goodbye Brian
732 | Chapter 29: Operator Overloading
www.it-ebooks.info
brian
Life
try finally
__del__ sys.stderr
__del__ __del__
gc
Chapter Summary
__enter__ __get__
__exit__ with __set__
__new__
Chapter Summary | 733
www.it-ebooks.info
__call__
__str__
Test Your Knowledge: Quiz
Test Your Knowledge: Answers __getitem__ __iter__ __next__
__iter__ __iter__
__getitem__ __str__
__repr__ print str __str__ __repr__ __str__ __str__ __repr__
print repr print
__getitem__ __getslice__ __iadd__
__add__ __radd__
734 | Chapter 29: Operator Overloading
str
str
www.it-ebooks.info
Test Your Knowledge: Answers | 735
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 30
Designing with Classes
Python and OOP
X.name X.method
method
X
737
www.it-ebooks.info
Overloading by Call Signatures (or Not)
class C: def meth(self, x): ... def meth(self, x, y, z): ...
def X = 1
class C: def meth(self, *args): if len(args) == 1: ... elif type(arg[0]) == int: ...
class C: def meth(self, x): x.operation()
738 | Chapter 30: Designing with Classes
X = 2 X
2
www.it-ebooks.info
OOP and Inheritance: “Is-a” Relationships
giveRaise
Employee __repr__ Employee Chef Server
work PizzaRobot
Chef
Employee
class Employee: def __init__(self, name, salary=0): self.name = name self.salary = salary def giveRaise(self, percent): self.salary = self.salary + (self.salary * percent) def work(self): print(self.name, "does stuff") def __repr__(self): return "" % (self.name, self.salary) class Chef(Employee): def __init__(self, name): Employee.__init__(self, name, 50000) def work(self): print(self.name, "makes food") class Server(Employee): def __init__(self, name): Employee.__init__(self, name, 40000) def work(self): print(self.name, "interfaces with customer") class PizzaRobot(Chef):
OOP and Inheritance: “Is-a” Relationships | 739
www.it-ebooks.info
def __init__(self, name): Chef.__init__(self, name) def work(self): print(self.name, "makes pizza") if __name__ == "__main__": bob = PizzaRobot('bob') print(bob) bob.work() bob.giveRaise(0.20) print(bob); print() for klass in Employee, Chef, Server, PizzaRobot: obj = klass(klass.__name__) obj.work()
bob bob Employee.giveRaise
PizzaRobot Chef Employee.__repr__
C:\python\examples> python employees.py bob makes pizza Employee does stuff Chef makes food Server interfaces with customer PizzaRobot makes pizza
for work work
OOP and Composition: “Has-a” Relationships
740 | Chapter 30: Designing with Classes
Employee bob
www.it-ebooks.info
from employees import PizzaRobot, Server class Customer: def __init__(self, name): self.name = name def order(self, server): print(self.name, "orders from", server) def pay(self, server): print(self.name, "pays for item to", server) class Oven: def bake(self): print("oven bakes") class PizzaShop: def __init__(self): self.server = Server('Pat') self.chef = PizzaRobot('Bob') self.oven = Oven() def order(self, name): customer = Customer(name) customer.order(self.server) self.chef.work() self.oven.bake() customer.pay(self.server) if __name__ == "__main__": scene = PizzaShop() scene.order('Homer') print('...') scene.order('Shaggy')
PizzaShop Oven PizzaShop order Customer Customer
Server
OOP and Composition: “Has-a” Relationships | 741
www.it-ebooks.info
C:\python\examples> python pizzashop.py Homer orders from Bob makes pizza oven bakes Homer pays for item to ... Shaggy orders from Bob makes pizza oven bakes Shaggy pays for item to
Stream Processors Revisited def processor(reader, converter, writer): while 1: data = reader.read() if not data: break data = converter(data) writer.write(data)
class Processor: def __init__(self, reader, writer): self.reader = reader self.writer = writer def process(self): while 1: data = self.reader.readline() if not data: break data = self.converter(data) self.writer.write(data) def converter(self, data): assert False, 'converter must be defined'
converter assert reader
742 | Chapter 30: Designing with Classes
writer
www.it-ebooks.info
from streams import Processor class Uppercase(Processor): def converter(self, data): return data.upper() if __name__ == '__main__': import sys obj = Uppercase(open('spam.txt'), sys.stdout) obj.process()
Uppercase
stdout C:\lp4e> type spam.txt spam Spam SPAM! C:\lp4e> python converters.py SPAM SPAM SPAM!
C:\lp4e> python >>> import converters >>> prog = converters.Uppercase(open('spam.txt'), open('spamup.txt', 'w')) >>> prog.process() C:\lp4e> type spamup.txt SPAM SPAM SPAM!
C:\lp4e> python >>> from converters import Uppercase >>> >>> class HTMLize: ... def write(self, line): ... print('%s' % line.rstrip()) ... >>> Uppercase(open('spam.txt'), HTMLize()).process() SPAM SPAM SPAM!
OOP and Composition: “Has-a” Relationships | 743
www.it-ebooks.info
Processor write convert Processor
Why You Will Care: Classes and Persistence pickle
pickle
shelve
import pickle object = someClass() file = open(filename, 'wb') pickle.dump(object, file) import pickle file = open(filename, 'rb') object = pickle.load(file)
744 | Chapter 30: Designing with Classes
shelve
www.it-ebooks.info
import shelve object = someClass() dbase = shelve.open('filename') dbase['key'] = object import shelve dbase = shelve.open('filename') object = dbase['key']
>>> from pizzashop import PizzaShop >>> shop = PizzaShop() >>> shop.server, shop.chef (, ) >>> import pickle >>> pickle.dump(shop, open('shopfile.dat', 'wb'))
shop
>>> import pickle >>> obj = pickle.load(open('shopfile.dat', 'rb')) >>> obj.server, obj.chef (, ) >>> obj.order('Sue') Sue orders from Bob makes pizza oven bakes Sue pays for item to
OOP and Delegation: “Wrapper” Objects
__getattr__ __getattr__
OOP and Delegation: “Wrapper” Objects | 745
www.it-ebooks.info
class wrapper: def __init__(self, object): self.wrapped = object def __getattr__(self, attrname): print('Trace:', attrname) return getattr(self.wrapped, attrname)
__getattr__ getattr getattr(X,N) X.N
N getattr(X,N) X.N
X.__dict__[N] __dict__ wrapper
wrapped >>> from trace import wrapper >>> x = wrapper([1,2,3]) >>> x.append(4) Trace: append >>> x.wrapped [1, 2, 3, 4] >>> x = wrapper({"a": 1, "b": 2}) >>> x.keys() Trace: keys ['a', 'b']
wrapped wrapper
746 | Chapter 30: Designing with Classes
www.it-ebooks.info
__getattr__ __repr__
__str__
__getattr__
Pseudoprivate Class Attributes
_X
__getattr__
__setattr__
Pseudoprivate Class Attributes | 747
www.it-ebooks.info
Name Mangling Overview class __X Spam
_Spam__X
class
self __meth
Spam
_Spam__meth self._Spam__X
self.__X
Why Use Pseudoprivate Attributes?
self self.attr = value
X class C1: def meth1(self): self.X = 88 def meth2(self): print(self.X)
class C2: def metha(self): self.X = 99 def methb(self): print(self.X)
748 | Chapter 30: Designing with Classes
www.it-ebooks.info
class C3(C1, C2): ... I = C3()
self.X self.X X
I.X
class C1: def meth1(self): def meth2(self): class C2: def metha(self): def methb(self):
self.__X = 88 print(self.__X) self.__X = 99 print(self.__X)
class C3(C1, C2): pass I = C3() I.meth1(); I.metha() print(I.__dict__) I.meth2(); I.methb()
X dir _C1__X
_C2__X
I
X
% python private.py {'_C2__X': 99, '_C1__X': 88} 88 99
I._C1__X = 77
class Super: def method(self): ... class Tool:
Pseudoprivate Class Attributes | 749
www.it-ebooks.info
def __method(self): ... def other(self): self.__method() class Sub1(Tool, Super): ... def actions(self): self.method() class Sub2(Tool): def __init__(self): self.method = 99
class
Sub1
Tool
Super Sub1 Sub2
__X Private
Private
Methods Are Objects: Bound or Unbound __call__
750 | Chapter 30: Designing with Classes
www.it-ebooks.info
self
self
self
class Spam: def doit(self, message): print(message)
object1 = Spam() object1.doit('hello world')
object.name object1 Spam.doit object1 = Spam() x = object1.doit x('hello world')
Methods Are Objects: Bound or Unbound | 751
www.it-ebooks.info
doit
object1 = Spam() t = Spam.doit t(object1, 'howdy')
self self.method self class Eggs: def m1(self, n): print(n) def m2(self): x = self.m1 x(42) Eggs().m2()
Unbound Methods are Functions in 3.0
C:\misc> c:\python30\python >>> class Selfless:
752 | Chapter 30: Designing with Classes
www.it-ebooks.info
... ... ... ... ... ... ... >>> >>> 9 >>> 9 >>> 7
def __init__(self, data): self.data = data def selfless(arg1, arg2): return arg1 + arg2 def normal(self, arg1, arg2): return self.data + arg1 + arg2 X = Selfless(2) X.normal(3, 4) Selfless.normal(X, 3, 4) Selfless.selfless(3, 4)
>>> X.selfless(3, 4) TypeError: selfless() takes exactly 2 positional arguments (3 given) >>> Selfless.normal(3, 4) TypeError: normal() takes exactly 3 positional arguments (2 given)
staticmethod self
Methods Are Objects: Bound or Unbound | 753
www.it-ebooks.info
Bound Methods and Other Callable Objects
>>> ... ... ... ... ... ... ... >>> >>> >>> >>> 4
class Number: def __init__(self, base): self.base = base def double(self): return self.base * 2 def triple(self): return self.base * 3 x = Number(2) y = Number(3) z = Number(4) x.double()
>>> acts = [x.double, y.double, y.triple, z.double] >>> for act in acts: ... print(act()) ... 4 6 9 8
>>> bound = x.double >>> bound.__self__, bound.__func__ (, ) >>> bound.__self__.base 2 >>> bound() 4
def __call__ >>> def square(arg): ... return arg ** 2 ... >>> class Sum: ... def __init__(self, val):
754 | Chapter 30: Designing with Classes
lambda
www.it-ebooks.info
... ... ... ... >>> ... ... ... ... ... >>> >>> >>>
self.val = val def __call__(self, arg): return self.val + arg class Product: def __init__(self, val): self.val = val def method(self, arg): return self.val * arg sobject = Sum(2) pobject = Product(3) actions = [square, sobject, pobject.method]
>>> for act in actions: ... print(act(5)) ... 25 7 15 >>> actions[-1](5) 15 >>> [act(5) for act in actions] [25, 7, 15] >>> list(map(lambda act: act(5), actions)) [25, 7, 15]
>>> class Negate: ... def __init__(self, val): ... self.val = -val ... def __repr__(self): ... return str(self.val) ... >>> actions = [square, sobject, pobject.method, Negate] >>> for act in actions: ... print(act(5)) ... 25 7 15 -5 >>> [act(5) for act in actions] [25, 7, 15, −5] >>> table = {act(5): act for act in actions} >>> for (key, value) in table.items(): ... print('{0:2} => {1}'.format(key, value)) ... -5 => 25 => 15 => 7 =>
Methods Are Objects: Bound or Unbound | 755
www.it-ebooks.info
__call__
Why You Will Care: Bound Methods and Callbacks
tkinter
Tkinter
def handler(): ...use globals for state... ... widget = Button(text='spam', command=handler)
command
lambda
class MyWidget: def handler(self): ...use self.attr for state... def makewidgets(self): b = Button(text='spam', command=self.handler)
self
MyGui.handler
self.handler self
handler
__call__
Multiple Inheritance: “Mix-in” Classes class
756 | Chapter 30: Designing with Classes
www.it-ebooks.info
class
self
Coding Mix-in Display Classes >>> class Spam: ... def __init__(self): ... self.data1 = "food" ... >>> X = Spam() >>> print(X)
__str__
__repr__
Multiple Inheritance: “Mix-in” Classes | 757
www.it-ebooks.info
AttrDisplay
__str__
Listing instance attributes with __dict__ ListInstance __str__ ListInstance
class ListInstance: """ Mix-in class that provides a formatted print() or str() of instances via inheritance of __str__, coded here; displays instance attrs only; self is the instance of lowest class; uses __X names to avoid clashing with client's attrs """ def __str__(self): return '' % ( self.__class__.__name__, id(self), self.__attrnames()) def __attrnames(self): result = '' for attr in sorted(self.__dict__): result += '\tname %s=%s\n' % (attr, self.__dict__ [attr]) retubrn result
ListInstance __class__ __name__ self.__class__.__name__
758 | Chapter 30: Designing with Classes
www.it-ebooks.info
__dict__
ListInstance
id
__attrnames
_ListInstance__attrnames self
ListInstance
__str__
>>> from lister import ListInstance >>> class Spam(ListInstance): ... def __init__(self): ... self.data1 = 'food' ... >>> x = Spam() >>> print(x)
str >>> str(x) '' >>> x
ListInstance ListInstance __str__
Multiple Inheritance: “Mix-in” Classes | 759
www.it-ebooks.info
from lister import * class Super: def __init__(self): self.data1 = 'spam' def ham(self): pass class Sub(Super, ListInstance): def __init__(self): Super.__init__(self) self.data2 = 'eggs' self.data3 = 42 def spam(self): pass if __name__ == '__main__': X = Sub() print(X)
Sub
Super
ListInstance Sub ListInstance
C:\misc> C:\python30\python testmixin.py
ListInstance
self
Lister import >>> import lister >>> class C(lister.ListInstance): pass ... >>> x = C() >>> x.a = 1; x.b = 2; x.c = 3 >>> print(x)
760 | Chapter 30: Designing with Classes
www.it-ebooks.info
ListInstance
__str__
__str__
Listing inherited attributes with dir Lister
dir
__dict__
class ListInherited: """ Use dir() to collect both instance attrs and names inherited from its classes; Python 3.0 shows more names than 2.6 because of the implied object superclass in the new-style class model; getattr() fetches inherited names not in self.__dict__; use __str__, not __repr__, or else this loops when printing bound methods! """ def __str__(self): return '' % ( self.__class__.__name__, id(self), self.__attrnames()) def __attrnames(self): result = '' for attr in dir(self): if attr[:2] == '__' and attr[-2:] == '__': result += '\tname %s=\n' % attr else: result += '\tname %s=%s\n' % (attr, getattr(self, attr)) return result
__X__ getattr getattr
Multiple Inheritance: “Mix-in” Classes | 761
www.it-ebooks.info
class Sub(Super, ListInherited):
C:\misc> c:\python26\python testmixin.py
object
C:\misc> c:\python30\python testmixin.py
__str__
__repr__
__repr__ __repr__ __repr__ __repr__
762 | Chapter 30: Designing with Classes
www.it-ebooks.info
__str__
__repr__
__repr__ isinstance
types.MethodType
Listing attributes per object in class trees
__class__
__bases__ __dicts__
class ListTree: """ Mix-in that returns an __str__ trace of the entire class tree and all its objects' attrs at and above self; run by print(), str() returns constructed string; uses __X attr names to avoid impacting clients; uses generator expr to recurse to superclasses; uses str.format() to make substitutions clearer """ def __str__(self): self.__visited = {} return ''.format( self.__class__.__name__, id(self), self.__attrnames(self, 0), self.__listclass(self.__class__, 4)) def __listclass(self, aClass, indent): dots = '.' * indent if aClass in self.__visited: return '\n{0}\n'.format( dots, aClass.__name__, id(aClass)) else: self.__visited[aClass] = True genabove = (self.__listclass(c, indent+4) for c in aClass.__bases__) return '\n{0}\n'.format( dots, aClass.__name__, id(aClass), self.__attrnames(aClass, indent), ''.join(genabove), dots) def __attrnames(self, obj, indent):
Multiple Inheritance: “Mix-in” Classes | 763
www.it-ebooks.info
spaces = ' ' * (indent + 4) result = '' for attr in sorted(obj.__dict__): if attr.startswith('__') and attr.endswith('__'): result += spaces + '{0}=\n'.format(attr) else: result += spaces + '{0}={1}\n'.format(attr, getattr(obj, attr)) return result
join format
%
return '' % (...) return ''.format(...)
class Sub(Super, ListTree):
C:\misc> c:\python26\python testmixin.py
764 | Chapter 30: Designing with Classes
www.it-ebooks.info
__visited
object C:\misc> c:\python30\python testmixin.py
id
__X__
C:\misc> c:\python26\python testmixin.py ...more omitted... ........', getattr(X, attr)) d => 4 a => 1 b => 2 __dict__ => {'d': 4}
getattr >>> for attr in list(getattr(X, '__dict__', [])) + getattr(X, '__slots__', []): ... print(attr, '=>', getattr(X, attr)) d => 4 a => 1 b => 2 __dict__ => {'d': 4}
Multiple __slot__ lists in superclasses __slots__ __slots__
__slots__ __dict__ __slots__
790 | Chapter 31: Advanced Class Topics
www.it-ebooks.info
__slots__ __dict__
__slots__
dir >>> ... ... >>> ... ... >>> >>> >>> (1,
class E: __slots__ = ['c', 'd'] class D(E): __slots__ = ['a', '__dict__'] X = D() X.a = 1; X.b = 2; X.c = 3 X.a, X.c 3)
>>> E.__slots__ ['c', 'd'] >>> D.__slots__ ['a', '__dict__'] >>> X.__slots__ ['a', '__dict__'] >>> X.__dict__ {'b': 2} >>> for attr in list(getattr(X, '__dict__', [])) + getattr(X, '__slots__', []): ... print(attr, '=>', getattr(X, attr)) ... b => 2 a => 1 __dict__ => {'b': 2} >>> dir(X) [...many names omitted... 'a', 'b', 'c', 'd']
__slots__
__dict__
Private
New-Style Class Extensions | 791
www.it-ebooks.info
Class Properties
__getattr__
__setattr__
property None class
name = property(...) obj.name __getattr__
>>> class classic: ... def __getattr__(self, name): ... if name == 'age': ... return 40 ... else: ... raise AttributeError ... >>> x = classic() >>> x.age 40 >>> x.name AttributeError
object >>> class newprops(object): ... def getage(self): ... return 40 ... age = property(getage, None, None, None) ... >>> x = newprops() >>> x.age 40 >>> x.name AttributeError: newprops instance has no attribute 'name'
792 | Chapter 31: Advanced Class Topics
www.it-ebooks.info
>>> class newprops(object): ... def getage(self): ... return 40 ... def setage(self, value): ... print('set age:', value) ... self._age = value ... age = property(getage, setage, None, None) ... >>> x = newprops() >>> x.age 40 >>> x.age = 42 set age: 42 >>> x._age 42 >>> x.job = 'trainer' >>> x.job 'trainer'
object
__setattr__
>>> class classic: ... def __getattr__(self, name): ... if name == 'age': ... return 40 ... else: ... raise AttributeError ... def __setattr__(self, name, value): ... print('set:', name, value) ... if name == 'age': ... self.__dict__['_age'] = value ... else: ... self.__dict__[name] = value ... >>> x = classic() >>> x.age 40 >>> x.age = 41 set: age 41 >>> x._age 41 >>> x.job = 'trainer' >>> x.job
New-Style Class Extensions | 793
www.it-ebooks.info
__getattr__
__setattr__
__getattr__
__setattr__
__getattribute__ and Descriptors __getattribute__ __getattr__ __getattr__ __setattr__ __get__
__getattribute__
Metaclasses
type
794 | Chapter 31: Advanced Class Topics
__set__
www.it-ebooks.info
Static and Class Methods
staticmethod classmethod staticmethod
Why the Special Methods?
self
self self
Static and Class Methods | 795
www.it-ebooks.info
self
Static Methods in 2.6 and 3.0
self
class Spam: numInstances = 0 def __init__(self): Spam.numInstances = Spam.numInstances + 1
796 | Chapter 31: Advanced Class Topics
www.it-ebooks.info
def printNumInstances(): print("Number of instances created: ", Spam.numInstances)
printNumInstances
C:\misc> c:\python26\python >>> from spam import Spam >>> a = Spam() >>> b = Spam() >>> c = Spam() >>> Spam.printNumInstances() TypeError: unbound method printNumInstances() must be called with Spam instance as first argument (got nothing instead) >>> a.printNumInstances() TypeError: printNumInstances() takes no arguments (1 given)
def
C:\misc> c:\python30\python >>> from spam import Spam >>> a = Spam() >>> b = Spam() >>> c = Spam() >>> Spam.printNumInstances() Number of instances created: 3 >>> a.printNumInstances() TypeError: printNumInstances() takes no arguments (1 given)
printNumInstances
Spam.printNumInstances() instance.printNumInstances()
Static and Class Methods | 797
www.it-ebooks.info
Static Method Alternatives
print def printNumInstances(): print("Number of instances created: ", Spam.numInstances) class Spam: numInstances = 0 def __init__(self): Spam.numInstances = Spam.numInstances + 1 >>> import spam >>> a = spam.Spam() >>> b = spam.Spam() >>> c = spam.Spam() >>> spam.printNumInstances() Number of instances created: >>> spam.Spam.numInstances 3
798 | Chapter 31: Advanced Class Topics
3
www.it-ebooks.info
class Spam: numInstances = 0 def __init__(self): Spam.numInstances = Spam.numInstances + 1 def printNumInstances(self): print("Number of instances created: ", Spam.numInstances) >>> from spam import Spam >>> a, b, c = Spam(), Spam(), Spam() >>> a.printNumInstances() Number of instances created: 3 >>> Spam.printNumInstances(a) Number of instances created: 3 >>> Spam().printNumInstances() Number of instances created: 4
Using Static and Class Methods
staticmethod
classmethod
class Methods: def imeth(self, x): print(self, x) def smeth(x): print(x) def cmeth(cls, x): print(cls, x) smeth = staticmethod(smeth) cmeth = classmethod(cmeth)
smeth
cmeth
class
def
Static and Class Methods | 799
www.it-ebooks.info
>>> obj = Methods() >>> obj.imeth(1) 1 >>> Methods.imeth(obj, 2) 2
staticmethod staticmethod >>> Methods.smeth(3) 3 >>> obj.smeth(4) 4
>>> Methods.cmeth(5) 5 >>> obj.cmeth(6) 6
Counting Instances with Static Methods
800 | Chapter 31: Advanced Class Topics
www.it-ebooks.info
class Spam: numInstances = 0 def __init__(self): Spam.numInstances += 1 def printNumInstances(): print("Number of instances:", Spam.numInstances) printNumInstances = staticmethod(printNumInstances)
>>> a = Spam() >>> b = Spam() >>> c = Spam() >>> Spam.printNumInstances() Number of instances: 3 >>> a.printNumInstances() Number of instances: 3
printNumInstances staticmethod class
class Sub(Spam): def printNumInstances(): print("Extra stuff...") Spam.printNumInstances() printNumInstances = staticmethod(printNumInstances) >>> a = Sub() >>> b = Sub() >>> a.printNumInstances() Extra stuff... Number of instances: 2 >>> Sub.printNumInstances() Extra stuff... Number of instances: 2 >>> Spam.printNumInstances() Number of instances: 2
>>> class Other(Spam): pass >>> c = Other() >>> c.printNumInstances() Number of instances: 3
Static and Class Methods | 801
www.it-ebooks.info
Counting Instances with Class Methods
class Spam: numInstances = 0 def __init__(self): Spam.numInstances += 1 def printNumInstances(cls): print("Number of instances:", cls.numInstances) printNumInstances = classmethod(printNumInstances)
printNumInstances
>>> a, b = Spam(), Spam() >>> a.printNumInstances() Number of instances: 2 >>> Spam.printNumInstances() Number of instances: 2
Spam.printNumInstances cls class Spam: numInstances = 0 def __init__(self): Spam.numInstances += 1 def printNumInstances(cls): print("Number of instances:", cls.numInstances, cls) printNumInstances = classmethod(printNumInstances) class Sub(Spam): def printNumInstances(cls): print("Extra stuff...", cls) Spam.printNumInstances() printNumInstances = classmethod(printNumInstances) class Other(Spam): pass
>>> x, y = Sub(), Spam() >>> x.printNumInstances() Extra stuff... Number of instances: 2
802 | Chapter 31: Advanced Class Topics
www.it-ebooks.info
>>> Sub.printNumInstances() Extra stuff... Number of instances: 2 >>> y.printNumInstances() Number of instances: 2
Sub Sub Sub
Spam Spam
>>> z = Other() >>> z.printNumInstances() Number of instances: 3
Other
Spam Spam Object
Spam
Spam
Counting instances per class with class methods
class Spam: numInstances = 0 def count(cls): cls.numInstances += 1 def __init__(self): self.count() count = classmethod(count) class Sub(Spam): numInstances = 0 def __init__(self): Spam.__init__(self) class Other(Spam): numInstances = 0
Static and Class Methods | 803
www.it-ebooks.info
>>> >>> >>> >>> (1, >>> (1,
x = Spam() y1, y2 = Sub(), Sub() z1, z2, z3 = Other(), Other(), Other() x.numInstances, y1.numInstances, z1.numInstances 2, 3) Spam.numInstances, Sub.numInstances, Other.numInstances 2, 3)
numInstances
Decorators and Metaclasses: Part 1 staticmethod
Function Decorator Basics def @
804 | Chapter 31: Advanced Class Topics
www.it-ebooks.info
class C: @staticmethod def meth(): ...
class C: def meth(): ... meth = staticmethod(meth)
staticmethod
classmethod class Spam: numInstances = 0 def __init__(self): Spam.numInstances = Spam.numInstances + 1 @staticmethod def printNumInstances(): print("Number of instances created: ", Spam.numInstances) a = Spam() b = Spam() c = Spam() Spam.printNumInstances() a.printNumInstances()
staticmethod
A First Function Decorator Example
Decorators and Metaclasses: Part 1 | 805
www.it-ebooks.info
__call__
class tracer: def __init__(self, func): self.calls = 0 self.func = func def __call__(self, *args): self.calls += 1 print('call %s to %s' % (self.calls, self.func.__name__)) self.func(*args) @tracer def spam(a, b, c): print(a, b, c) spam(1, 2, 3) spam('a', 'b', 'c') spam(4, 5, 6)
spam spam
tracer __call__
*name
spam tracer spam call 1 to spam 1 2 3 call 2 to spam a b c call 3 to spam 4 5 6
__call__ def
806 | Chapter 31: Advanced Class Topics
tracer
www.it-ebooks.info
Class Decorators and Metaclasses class
def decorator(aClass): ... @decorator class C: ...
def decorator(aClass): ... class C: ... C = decorator(C)
def count(aClass): aClass.numInstances = 0 return aClass @count class Spam: ... @count class Sub(Spam): ... @count class Other(Spam): ...
type
class
class Meta(type): def __new__(meta, classname, supers, classdict): ... class C(metaclass=Meta): ...
Decorators and Metaclasses: Part 1 | 807
www.it-ebooks.info
class class C: __metaclass__ = Meta ...
__new__
__init__
For More Details
Class Gotchas
Changing Class Attributes Can Have Side Effects
808 | Chapter 31: Advanced Class Topics
type
www.it-ebooks.info
a X.a
X >>> ... ... >>> >>> 1 >>> 1
class X: a = 1 I = X() I.a X.a
class
>>> >>> 2 >>> >>> 2
X.a = 2 I.a J = X() J.a
class X: pass class Y: pass X.a X.b X.c Y.a
= = = =
1 2 3 X.a + X.b + X.c
for X.i in range(Y.a): print(X.i)
X
Y
Class Gotchas | 809
www.it-ebooks.info
class Record: pass X = Record() X.name = 'bob' X.job = 'Pizza maker'
Changing Mutable Class Attributes Can Have Side Effects, Too
>>> class C: ... shared = [] ... def __init__(self): ... self.perobj = [] ... >>> x = C() >>> y = C() >>> y.shared, y.perobj ([], []) >>> x.shared.append('spam') >>> x.perobj.append('spam') >>> x.shared, x.perobj (['spam'], ['spam']) >>> y.shared, y.perobj (['spam'], []) >>> C.shared ['spam']
x.shared.append('spam') x.shared = 'spam'
810 | Chapter 31: Advanced Class Topics
www.it-ebooks.info
Multiple Inheritance: Order Matters class
Super
__str__
class ListTree: def __str__(self): ... class Super: def __str__(self): ... class Sub(ListTree, Super): x = Sub()
ListTree Sub
Super
class
ListTree
__str__ tkinter.Button Super
__str__
ListTree Super
ListTree
class Sub class ListTree: def __str__(self): ... def other(self): ... class Super: def __str__(self): ... def other(self): ... class Sub(ListTree, Super): other = Super.other def __init__(self): ... x = Sub()
other Super.other ListTree.other Super ListTree
Sub
class
Sub.other Sub.other other
Class Gotchas | 811
www.it-ebooks.info
class Sub(Super, ListTree): __str__ = Lister.__str__
__X
__str__
other
ListTree __other
Methods, Classes, and Nested Scopes
class
generate Spam generate method
Spam Spam generate
def generate(): class Spam: count = 1 def method(self): print(Spam.count) return Spam() generate().method() C:\python\examples> python nester.py ...error text omitted...
812 | Chapter 31: Advanced Class Topics
method
www.it-ebooks.info
Print(Spam.count) NameError: Spam
def
def
def
def def self self.count
class Spam.count
count Spam
global
method Spam
def generate(): global Spam class Spam: count = 1 def method(self): print(Spam.count) return Spam() generate().method()
Spam global method
generate
Spam def generate(): return Spam() class Spam: count = 1 def method(self): print(Spam.count) generate().method()
Spam method
__class__
def generate(): class Spam:
Class Gotchas | 813
www.it-ebooks.info
count = 1 def method(self): print(self.__class__.count) return Spam() generate().method()
Delegation-Based Classes in 3.0: __getattr__ and built-ins __getattr__
__str__
“Overwrapping-itis”
814 | Chapter 31: Advanced Class Topics
__getattr__
www.it-ebooks.info
Chapter Summary
Test Your Knowledge: Quiz
Test Your Knowledge: Answers
Test Your Knowledge: Answers | 815
www.it-ebooks.info
object
type __getattr__ __slots__ self
Test Your Knowledge: Part VI Exercises
Adder add ListAdder add
816 | Chapter 31: Advanced Class Topics
add(self, x, y) Adder
www.it-ebooks.info
DictAdder add
add Adder self.data Y
+
__add__ X.add(X.data,Y)
add
X +
add add(self,y) self.data + y add Mylist + append
sort
Mylist
start[:] Mylist Mylist
Mylist +
Mylist stdout MylistSub MylistSub
Mylist Mylist
MylistSub
+ stdout
Test Your Knowledge: Part VI Exercises | 817
www.it-ebooks.info
self Meta stdout
Meta
__getattr__
&
|
for
*args &
__add__
__getattr__
__bases__ __bases__
>> from zoo import Cat, Hacker >>> spot = Cat() >>> spot.reply() meow >>> data = Hacker() >>> data.reply() Hello world!
Scene Clerk Parrot
action
Customer line
line % python >>> import parrot >>> parrot.Scene().action() customer: "that's one ex-bird!"
820 | Chapter 31: Advanced Class Topics
line
www.it-ebooks.info
clerk: "no it isn't..." parrot: None
Why You Will Care: OOP by the Masters
Test Your Knowledge: Part VI Exercises | 821
www.it-ebooks.info
822 | Chapter 31: Advanced Class Topics
www.it-ebooks.info
PART VII
Exceptions and Tools
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 32
Exception Basics
try except try finally raise assert with as
Why Use Exceptions?
825
www.it-ebooks.info
try
Exception Roles
try try try
826 | Chapter 32: Exception Basics
www.it-ebooks.info
try finally
Exceptions: The Short Story
Default Exception Handler >>> def fetcher(obj, index): ... return obj[index] ...
>>> x = 'spam' >>> fetcher(x, 3) 'm'
obj[index] IndexError
next(G) try
Exceptions: The Short Story | 827
www.it-ebooks.info
>>> fetcher(x, 4) Traceback (most recent call last): File "", line 1, in File "", line 2, in fetcher IndexError: string index out of range
>>> fetcher(x, 4) Traceback (most recent call last): File "", line 1, in fetcher(x, 4) File "", line 2, in fetcher return obj[index] IndexError: string index out of range
Catching Exceptions try >>> try: ... fetcher(x, 4) ... except IndexError: ... print('got exception') ... got exception >>>
except try except try
828 | Chapter 32: Exception Basics
www.it-ebooks.info
>>> def catcher(): ... try: ... fetcher(x, 4) ... except IndexError: ... print('got exception') ... print('continuing') ... >>> catcher() got exception continuing >>>
try
Raising Exceptions
raise
>>> try: ... raise IndexError ... except IndexError: ... print('got exception') ... got exception
>>> raise IndexError Traceback (most recent call last): File "", line 1, in IndexError
assert raise >>> assert False, 'Nobody expects the Spanish Inquisition!' Traceback (most recent call last): File "", line 1, in AssertionError: Nobody expects the Spanish Inquisition!
Exceptions: The Short Story | 829
www.it-ebooks.info
User-Defined Exceptions raise
Exception >>> ... ... >>> ... ... >>> ... ... ... ... got >>>
class Bad(Exception): pass def doomed(): raise Bad() try: doomed() except Bad: print('got Bad') Bad
Termination Actions try
finally except
try finally try
>>> try: ... fetcher(x, 3) ... finally: ... print('after fetch') ... 'm' after fetch >>>
try
finally try print try
fetcher(x, 3) print('after fetch')
print try
830 | Chapter 32: Exception Basics
try finally finally
www.it-ebooks.info
>>> def after(): ... try: ... fetcher(x, 4) ... finally: ... print('after fetch') ... print('after try?') ... >>> after() after fetch Traceback (most recent call last): File "", line 1, in File "", line 3, in after File "", line 2, in fetcher IndexError: string index out of range >>>
try finally finally finally
try
>>> def after(): ... try: ... fetcher(x, 3) ... finally: ... print('after fetch') ... print('after try?') ... >>> after() after fetch after try? >>>
try except try finally try try except try finally
except
finally
try
finally
except try finally
with as
Exceptions: The Short Story | 831
www.it-ebooks.info
>>> with open('lumberjack.txt', 'w') as file: ... file.write('The larch!\n')
try finally with as
Why You Will Care: Error Checks
doStuff() { if (doFirstThing() == ERROR) return ERROR; if (doNextThing() == ERROR) return ERROR; ... return doLastThing(); } main() { if (doStuff() == ERROR) badEnding(); else goodEnding(); }
def doStuff(): doFirstThing() doNextThing() ... doLastThing() if __name__ == '__main__': try: doStuff() except: badEnding() else: goodEnding()
832 | Chapter 32: Exception Basics
www.it-ebooks.info
Chapter Summary
try try raise
assert
with as
try
Test Your Knowledge: Quiz
Test Your Knowledge: Answers
Test Your Knowledge: Answers | 833
www.it-ebooks.info
try except
raise
assert
try finally with as
834 | Chapter 32: Exception Basics
www.it-ebooks.info
CHAPTER 33
Exception Coding Details
try raise assert
with
try
except
finally else with
raise
except
The try/except/else Statement try except else
try finally except
finally
try try try
835
www.it-ebooks.info
except
else try except
else
try:
except : except (name2, name3): except as : except: else:
name1
name4
try except else
try raise try
try try try try try
except try
except
except try
except try
try else try except else
try try
except try try try 836 | Chapter 33: Exception Coding Details
try
www.it-ebooks.info
try Statement Clauses try
try except
finally
else except
else
except finally
else
finally except try
finally else except finally
try
else
Clause form
Interpretation
except:
Catch all (or all other) exception types.
except name:
Catch a specific exception only.
except name as value:
Catch the listed exception and its instance.
except (name1, name2):
Catch any of the listed exceptions.
except (name1, name2) as value:
Catch any listed exception and its instance.
else:
Run if no exceptions are raised.
finally:
Always perform this block.
as value
except
raise
except: try
except
except (e1, e2, e3): try
except
except except try:
action() except NameError: ... except IndexError: ...
The try/except/else Statement | 837
www.it-ebooks.info
except KeyError: ... except (AttributeError, TypeError, SyntaxError): ... else: ...
try except try
action except
else
action except except
try:
action() except NameError: ... except IndexError: ... except: ... else: ...
except try try:
action() except: ...
except
Exception try:
action() except Exception: ...
838 | Chapter 33: Exception Coding Details
except
www.it-ebooks.info
empty
except E as V: except E, V:
except (E1, E2):
as
as
V
except
The try else Clause else try try: ...run code... except IndexError: ...handle exception...
else
else try
try: ...run code... except IndexError: ...handle exception... else: ...no exception occurred...
else
try
try: ...run code... ...no exception occurred... except IndexError: ...handle exception...
IndexError
try
The try/except/else Statement | 839
www.it-ebooks.info
try else except
try else
Example: Default Behavior try
def gobad(x, y): return x / y def gosouth(x): print(gobad(x, 0)) gosouth(1)
% python bad.py Traceback (most recent call last): File "bad.py", line 7, in gosouth(1) File "bad.py", line 5, in gosouth print(gobad(x, 0)) File "bad.py", line 2, in gobad return x / y ZeroDivisionError: int division or modulo by zero
return
840 | Chapter 33: Exception Coding Details
www.it-ebooks.info
SyntaxError
try pdb
Example: Catching Built-in Exceptions
try TypeError + def kaboom(x, y): print(x + y) try:
kaboom([0,1,2], "spam") except TypeError: print('Hello world!') print('resuming here')
kaboom
try
except try
% python kaboom.py Hello world! resuming here
try kaboom
The try/except/else Statement | 841
www.it-ebooks.info
The try/finally Statement try finally
try try
try try:
finally:
try try try finally
try try
finally try
try
finally except finally finally
try finally
finally
try
else
try finally
finally try
except
else
finally
with finally
842 | Chapter 33: Exception Coding Details
except
www.it-ebooks.info
Example: Coding Termination Actions with try/finally try finally class MyError(Exception): pass def stuff(file): raise MyError() file = open('data', 'w') try: stuff(file) finally: file.close() print('not reached')
try finally
try
try finally finally
try
finally try try finally try
The try/finally Statement | 843
www.it-ebooks.info
Unified try/except/finally try finally except else finally
except
else
finally except
else
Exception1
handler1
try: main-action except Exception1: handler1 except Exception2: handler2 ... else: else-block finally: finally-block
main-action except Exception2 handler2 else-block finally-block finally-block else-block finally finally-block
finallytry
block finally try finally
844 | Chapter 33: Exception Coding Details
www.it-ebooks.info
finally try
Unified try Statement Syntax try
except
finally
try -> except -> else -> finally
else
finally except
except
except else else
try finally
try: statements except [type [as value]]: statements [except [type [as value]]: statements]* [else: statements] [finally: statements] try: statements finally: statements
else except finally finally else
except else except
except
raise
Combining finally and except by Nesting finally try
try except
try
except try finally
Unified try/except/finally | 845
www.it-ebooks.info
try:
try: main-action except Exception1: handler1 except Exception2: handler2 ... else: no-error finally: cleanup
finally try else
except
finally
Unified try Example try print sep = '-' * 32 + '\n' print(sep + 'EXCEPTION RAISED AND CAUGHT') try: x = 'spam'[99] except IndexError: print('except run') finally: print('finally run') print('after run') print(sep + 'NO EXCEPTION RAISED') try: x = 'spam'[3] except IndexError: print('except run') finally: print('finally run') print('after run') print(sep + 'NO EXCEPTION RAISED, WITH ELSE') try:
846 | Chapter 33: Exception Coding Details
www.it-ebooks.info
x = 'spam'[3] except IndexError: print('except run') else: print('else run') finally: print('finally run') print('after run') print(sep + 'EXCEPTION RAISED BUT NOT CAUGHT') try: x = 1 / 0 except IndexError: print('except run') finally: print('finally run') print('after run')
print
c:\misc> C:\Python30\python mergedexc.py -------------------------------EXCEPTION RAISED AND CAUGHT except run finally run after run -------------------------------NO EXCEPTION RAISED finally run after run -------------------------------NO EXCEPTION RAISED, WITH ELSE else run finally run after run -------------------------------EXCEPTION RAISED BUT NOT CAUGHT finally run Traceback (most recent call last): File "mergedexc.py", line 36, in x = 1 / 0 ZeroDivisionError: int division or modulo by zero
Unified try/except/finally | 847
www.it-ebooks.info
The raise Statement raise raise
raise raise raise raise
raise raise
raise
raise IndexError raise IndexError()
raise IndexError exc = IndexError() raise exc excs = [IndexError, TypeError] raise excs[0]
try
except name as X: raise
try:
... except IndexError as X: ...
as
try
848 | Chapter 33: Exception Coding Details
X
www.it-ebooks.info
class MyExc(Exception): pass ... raise MyExc('spam') ... try: ... except MyExc as X: print(X.args)
except try raise
Propagating Exceptions with raise raise
>>> try: ... raise IndexError('spam') ... except IndexError: ... print('propagating') ... raise ... propagating Traceback (most recent call last): File "", line 2, in IndexError: spam
raise
Python 3.0 Exception Chaining: raise from raise
from
raise exception from otherexception
from __cause__ >>> try: ... 1 / 0 ... except Exception as E:
The raise Statement | 849
www.it-ebooks.info
... raise TypeError('Bad!') from E ... Traceback (most recent call last): File "", line 2, in ZeroDivisionError: int division or modulo by zero The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 4, in TypeError: Bad!
__context__
raise Exc, Args raise Exc(Args)
raise Exc
raise
Exc()
The assert Statement assert raise assert
raise
assert ,
data
if __debug__: if not : raise AssertionError()
AssertionError
try assert
-O AssertionError
850 | Chapter 33: Exception Coding Details
__debug__
www.it-ebooks.info
True
-O
python –O
main.py
Example: Trapping Constraints (but Not Errors!) assert def f(x): assert x < 0, 'x must be negative' return x ** 2 % python >>> import asserter >>> asserter.f(1) Traceback (most recent call last): File "", line 1, in File "asserter.py", line 2, in f assert x < 0, 'x must be negative' AssertionError: x must be negative
assert assert def reciprocal(x): assert x != 0 return 1 / x
assert assert assert
with/as Context Managers with as import from __future__ import with_statement
with/as Context Managers | 851
www.it-ebooks.info
with as
try
finally try finally
with
Basic Usage with with expression [as variable]: with-block
expression variable
as
variable expression
expression variable
expression
with-block
with with with open(r'C:\misc\data') as myfile: for line in myfile: print(line) ...more code here...
open myfile for with
myfile
with myfile
for with
852 | Chapter 33: Exception Coding Details
www.it-ebooks.info
try/finally myfile = open(r'C:\misc\data') try: for line in myfile: print(line) ...more code here... finally: myfile.close()
with lock = threading.Lock() with lock: ...access shared resources...
decimal
with decimal.localcontext() as ctx: ctx.prec = 2 x = decimal.Decimal('1.00') / decimal.Decimal('3.00')
try finally
The Context Management Protocol with with
with
with/as Context Managers | 853
www.it-ebooks.info
__enter__
__exit__ __enter__ as with
with
__exit__(type, value, traceback)
sys.exc_info
with with type value
__exit__ traceback
None with
with
class TraceBlock: def message(self, arg): print('running', arg) def __enter__(self): print('starting with block') return self def __exit__(self, exc_type, exc_value, exc_tb): if exc_type is None: print('exited normally\n') else: print('raise an exception!', exc_type) return False with TraceBlock() as action: action.message('test 1') print('reached') with TraceBlock() as action: action.message('test 2') raise TypeError print('not reached')
__exit__
False
return
None False
self
__enter__ as with
__enter__
__exit__
854 | Chapter 33: Exception Coding Details
www.it-ebooks.info
% python withas.py starting with block running test 1 reached exited normally starting with block running test 2 raise an exception! Traceback (most recent call last): File "withas.py", line 20, in raise TypeError TypeError
contextlib try finally
with
with open('data') as fin, open('res', 'w') as fout: for line in fin: if 'some key' in line: fout.write(line)
with with A() as a, B() as b: ...statements...
with A() as a: with B() as b: ...statements...
Chapter Summary try assert
raise
with
Chapter Summary | 855
www.it-ebooks.info
Test Your Knowledge: Quiz try try raise assert with as
Test Your Knowledge: Answers try
try
try except else
try finally except finally finally
try
raise raise assert
AssertionError raise
if
with as try finally
856 | Chapter 33: Exception Coding Details
www.it-ebooks.info
CHAPTER 34
Exception Objects
try try
857
www.it-ebooks.info
BaseException Exception
Exceptions: Back to the Future try
String Exceptions Are Right Out!
is C:\misc> C:\Python25\python >>> myexc = "My exception string" >>> try: ... raise myexc ... except myexc: ... print('caught') ... caught
858 | Chapter 34: Exception Objects
==
raise
www.it-ebooks.info
Class-Based Exceptions
except
except
is
except try
try
except except
except
Coding Exceptions Classes General Specific1 General
Specific2
Specific1
Specific2
General class General(Exception): pass class Specific1(General): pass
Exceptions: Back to the Future | 859
www.it-ebooks.info
class Specific2(General): pass def raiser0(): X = General() raise X def raiser1(): X = Specific1() raise X def raiser2(): X = Specific2() raise X for func in (raiser0, raiser1, raiser2): try: func() except General: import sys print('caught:', sys.exc_info()[0]) C:\python30> python classexc.py caught: caught: caught:
pass Exception
Exception
raise raise raise
raise
try try General
860 | Chapter 34: Exception Objects
General
www.it-ebooks.info
sys.exc_info
except sys.exc_info __class__ sys.exc_info
except
except __class__
class General(Exception): pass class Specific1(General): pass class Specific2(General): pass def raiser0(): raise General() def raiser1(): raise Specific1() def raiser2(): raise Specific2() for func in (raiser0, raiser1, raiser2): try: func() except General as X: print('caught:', X.__class__)
__class__ sys.exc_info
except
sys.exc_info __class__
Why Exception Hierarchies? except try:
func() except (General, Specific1, Specific2): ...
Why Exception Hierarchies? | 861
www.it-ebooks.info
except
class Divzero(Exception): pass class Oflow(Exception): pass def func(): ... raise Divzero()
try
import mathlib try:
mathlib.func(...) except (mathlib.Divzero, mathlib.Oflow): ...handle and recover...
class Divzero(Exception): pass class Oflow(Exception): pass class Uflow(Exception): pass
try:
mathlib.func(...) except (mathlib.Divzero, mathlib.Oflow, mathlib.Uflow): ...handle and recover...
862 | Chapter 34: Exception Objects
www.it-ebooks.info
try
except
try:
mathlib.func(...) except: ...handle and recover...
try
class NumErr(Exception): pass class Divzero(NumErr): pass class Oflow(NumErr): pass ... def func(): ... raise DivZero()
except
except
Why Exception Hierarchies? | 863
www.it-ebooks.info
import mathlib ... try: mathlib.func(...) except mathlib.NumErr: ...report and recover...
... class Uflow(NumErr): pass
Built-in Exception Classes
SyntaxError builtins __builtin__ exceptions BaseException Exception str print
864 | Chapter 34: Exception Objects
www.it-ebooks.info
args Exception BaseException SystemExit GeneratorExit BaseException
KeyboardInterrupt Exception
try
Exception except
try ArithmeticError
Exception OverflowError ArithmeticError
exceptions help >>> import exceptions >>> help(exceptions) ...lots of text omitted...
Built-in Exception Categories OverflowError
ArithmeticError ZeroDivisionError
ArithmeticError
try OverflowError Exception except try:
action() except Exception: ...handle all application exceptions... else: ...handle no-exception case...
Built-in Exception Classes | 865
www.it-ebooks.info
Exception except
Default Printing and State
args
>>> raise IndexError Traceback (most recent call last): File "", line 1, in IndexError >>> raise IndexError('spam') Traceback (most recent call last): File "", line 1, in IndexError: spam >>> I = IndexError('spam') >>> I.args ('spam',)
>>> class E(Exception): pass ... >>> try: ... raise E('spam') ... except E as X: ... print(X, X.args) ... spam ('spam',)
866 | Chapter 34: Exception Objects
www.it-ebooks.info
>>> try: ... raise E('spam', 'eggs', 'ham') ... except E as X: ... print(X, X.args) ... ('spam', 'eggs', 'ham') ('spam', 'eggs', 'ham')
__str__ str(X) + "string"
__str__
__init__
Exception
Custom Print Displays >>> class MyBad(Exception): pass ... >>> try: ... raise MyBad('Sorry--my mistake!') ... except MyBad as X: ... print(X) ... Sorry--my mistake!
>>> raise MyBad('Sorry--my mistake!') Traceback (most recent call last): File "", line 1, in __main__.MyBad: Sorry--my mistake!
__repr__ __str__
>>> class MyBad(Exception): ... def __str__(self): ... return 'Always look on the bright side of life...' ... >>> try: ... raise MyBad() ... except MyBad as X: ... print(X)
Custom Print Displays | 867
www.it-ebooks.info
... Always look on the bright side of life... >>> raise MyBad() Traceback (most recent call last): File "", line 1, in __main__.MyBad: Always look on the bright side of life...
__str__ __str__ __repr__
__str__ __repr__
__str__
Custom Data and Behavior args
Providing Exception Details raise try try try
try as except
868 | Chapter 34: Exception Objects
www.it-ebooks.info
>>> class FormatError(Exception): ... def __init__(self, line, file): ... self.line = line ... self.file = file ... >>> def parser(): ... raise FormatError(42, file='spam.txt') ... >>> try: ... parser() ... except FormatError as X: ... print('Error at', X.file, X.line) ... Error at spam.txt 42
except
X
>>> class FormatError(Exception): pass ... >>> def parser(): ... raise FormatError(42, 'spam.txt') ... >>> try: ... parser() ... except FormatError as X: ... print('Error at:', X.args[0], X.args[1]) ... Error at: 42 spam.txt
Providing Exception Methods
class FormatError(Exception): logfile = 'formaterror.txt' def __init__(self, line, file): self.line = line self.file = file
sys.exc_info() except sys.exc_info
Custom Data and Behavior | 869
www.it-ebooks.info
def logerror(self): log = open(self.logfile, 'a') print('Error at', self.file, self.line, file=log) def parser(): raise FormatError(40, 'spam.txt') try:
parser() except FormatError as exc: exc.logerror()
C:\misc> C:\Python30\python parse.py C:\misc> type formaterror.txt Error at spam.txt 40
line
Chapter Summary
try
870 | Chapter 34: Exception Objects
logerror file
www.it-ebooks.info
Test Your Knowledge: Quiz
Test Your Knowledge: Answers BaseException
Exception
args
__str__ print
str
Test Your Knowledge: Answers | 871
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 35
Designing with Exceptions
Nesting Exception Handlers try try try
try
try try
try
except
try try try try
except try
try
except try try raise func1
func2 func1 873
www.it-ebooks.info
finally
try finally try
finally try try finally try except
try except finally
874 | Chapter 35: Designing with Exceptions
www.it-ebooks.info
Example: Control-Flow Nesting action2 action1
action2
try
def action2(): print(1 + []) def action1(): try: action2() except TypeError: print('inner try') try:
action1() except TypeError: print('outer try') % python nestexc.py inner try
action1
try try
action2 action1
TypeError try
try
except
action1
Example: Syntactic Nesting try except finally try try:
try:
action2() except TypeError: print('inner try') except TypeError: print('outer try')
Nesting Exception Handlers | 875
www.it-ebooks.info
try finally >>> try: ... try: ... raise IndexError ... finally: ... print('spam') ... finally: ... print('SPAM') ... spam SPAM Traceback (most recent call last): File "", line 3, in IndexError
def raise1(): raise IndexError def noraise(): return def raise2(): raise SyntaxError for func in (raise1, noraise, raise2): print('\n', func, sep='') try: try: func() except IndexError: print('caught IndexError') finally: print('finally run')
finally except try % python except-finally.py caught IndexError finally run finally run finally run
876 | Chapter 35: Designing with Exceptions
finally
www.it-ebooks.info
Traceback (most recent call last): File "except-finally.py", line 9, in func() File "except-finally.py", line 3, in raise2 def raise2(): raise SyntaxError SyntaxError: None
except
finally
try
Exception Idioms Exceptions Aren’t Always Errors input sys.stdin
EOFError raw_input
input
EOFError input
try
while True: try: line = input() except EOFError: break else: ...process next line here...
sys.exit() SystemExit Key boardInterrupt
warnings
Exception Idioms | 877
www.it-ebooks.info
Functions Can Signal Conditions with raise try except else if else class Found(Exception): pass def searcher(): if ...success...: raise Found() else: return try:
searcher() except Found: ...success... else: ...failure...
class Failure(Exception): pass def searcher(): if ...success...: return ...founditem... else: raise Failure() try:
item = searcher() except Failure: ...report... else: ...use item here...
Closing Files and Server Connections
878 | Chapter 35: Designing with Exceptions
www.it-ebooks.info
try finally myfile = open(r'C:\misc\script', 'w') try: ...process myfile... finally: myfile.close()
with as with open(r'C:\misc\script', 'w') as myfile: ...process myfile...
try finally
try finally try finally try finally
Debugging with Outer try Statements try except sys.exc_info
sys
sys.exc_info
Exception Idioms | 879
www.it-ebooks.info
try:
...run program... except: import sys print('uncaught!', sys.exc_info()[0], sys.exc_info()[1])
Running In-Process Tests import sys log = open('testlog', 'a') from testapi import moreTests, runNextTest, testName def testdriver(): while moreTests(): try: runNextTest() except: print('FAILED', testName(), sys.exc_info()[:2], file=log) else: print('PASSED', testName(), file=log) testdriver()
testdriver
testapi try except sys.exc_info
else
os.system
os.popen
doctest
880 | Chapter 35: Designing with Exceptions
www.it-ebooks.info
More on sys.exc_info sys.exc_info except try:
... except:
None (type, value, traceback) type value traceback traceback
sys.exc_info
__class__
as
sys.exc_info
except try:
... except General as instance:
try:
... except General as instance:
Exception Idioms | 881
www.it-ebooks.info
sys.exc_type sys.exc_value
sys.exc_info()
Exception Design Tips and Gotchas except
try
What Should Be Wrapped try try
try
try try
try finally with as
try
finally
try
try try
882 | Chapter 35: Designing with Exceptions
www.it-ebooks.info
try
Catching Too Much: Avoid Empty except and Exception except try try
def func(): try: ... except: ... try: func() except IndexError: ...
sys.exit(statuscode) SystemExit try finally try
except
import sys def bye(): sys.exit(40) try: bye() except: print('got it')
os._exit try except
try finally
Exception Design Tips and Gotchas | 883
www.it-ebooks.info
print('continuing...') % python exiter.py got it continuing...
Exception
SystemExit
try:
bye() except Exception: ...
except Exception except
Exception
mydictionary = {...} ... try: x = myditctionary['spam'] except: x = None ...continue here with x...
myditctionary mydictionary
NameError
Exception
except
except Exception except KeyError:
884 | Chapter 35: Designing with Exceptions
www.it-ebooks.info
Catching Too Little: Use Class-Based Categories try
MyExcept1
MyExcept2 MyExcept3
try:
... except (MyExcept1, MyExcept2): ... else: ...
except try:
... except SuccessCategoryName: ... else: ...
try
Core Language Summary
Core Language Summary | 885
www.it-ebooks.info
The Python Toolset
Category
Examples
Object types
Lists, dictionaries, files, strings
Functions
len, range, open
Exceptions
IndexError, KeyError
Modules
os, tkinter, pickle, re
Attributes
__dict__, __name__, __class__
Peripheral tools
NumPy, SWIG, Jython, IronPython, Django, etc.
886 | Chapter 35: Designing with Exceptions
www.it-ebooks.info
Development Tools for Larger Projects
help
unittest __name__ == '__main__' unittest
doctest doctest
Core Language Summary | 887
www.it-ebooks.info
doctest doctest doctest
time
timeit
profile
profile
profile pstats profile.run('code')
profile
python -m profile main.py args... cProfile profile
pdb
888 | Chapter 35: Designing with Exceptions
www.it-ebooks.info
pdb pdb.run("main()") python -m pdb main.py args... pdb.pm()
print
distutils
-O
Core Language Summary | 889
www.it-ebooks.info
from *
Chapter Summary
890 | Chapter 35: Designing with Exceptions
__all__ _X __name__ == '__main__'
www.it-ebooks.info
Test Your Knowledge: Quiz
Test Your Knowledge: Answers
Test Your Knowledge: Part VII Exercises
try except
oops
IndexError try except oops KeyError IndexError oops
KeyError
IndexError oops MyError try IndexError
safe(func, *args) *name exc_info oops oops
sys
safe safe
safe print_exc
traceback
Test Your Knowledge: Part VII Exercises | 891
www.it-ebooks.info
892 | Chapter 35: Designing with Exceptions
www.it-ebooks.info
PART VIII
Advanced Topics
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 36
Unicode and Byte Strings
str
895
www.it-ebooks.info
String Changes in 3.0 str unicode bytearray
str bytearray
struct bytes
str
sys.getdefaultencoding()
896 | Chapter 36: Unicode and Byte Strings
bytes
www.it-ebooks.info
String Basics
Character Encoding Schemes
'a' ord
97 chr
>>> ord('a') 97 >>> hex(97) '0x61' >>> chr(97) 'a'
>>> 0xC4 196 >>> chr(196) 'Ä'
String Basics | 897
www.it-ebooks.info
encodings help(encodings)
898 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
Python’s String Types
str unicode unicode str
str bytes bytearray
bytes bytearray
str
str str
unicode
String Basics | 899
www.it-ebooks.info
struct bytes str unicode bytes bytes str
re
bytes bytes
int
bytes
list bytes isalpha bytes bytearray bytearray
bytes str bytes append extend bytearray str
str
bytes
str bytearray
str
str
bytes str unicode
unicode str
bytes
bytearray
Text and Binary Files str bytes
900 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
str str
b open bytes bytes bytearray str
bytes str bytes
bytes bytearray
str open b bytes b rb wb
str rb+
bytes r w+
rt
str
String Basics | 901
www.it-ebooks.info
Python 3.0 Strings in Action str bytes str bytes(X)
str(X) '...' bytes bytes
b'...'
Literals and Basic Properties str bytes
open b'xxx'
B'xxx'
bytes
bytearray
bytearray 'xxx' "xxx" str b'...' C:\misc> c:\python30\python >>> B = b'spam' >>> S = 'eggs' >>> type(B), type(S) (, ) >>> B b'spam' >>> S 'eggs'
902 | Chapter 36: Unicode and Byte Strings
b
B
bytes r'...'
www.it-ebooks.info
bytes >>> B[0], S[0] (115, 'e') >>> B[1:], S[1:] (b'pam', 'ggs') >>> list(B), list(S) ([115, 112, 97, 109], ['e', 'g', 'g', 's'])
bytes
str str bytes
bytearray bytes
bytes
>>> B[0] = 'x' TypeError: 'bytes' object does not support item assignment >>> S[0] = 'x' TypeError: 'str' object does not support item assignment >>> B = B""" ... xxxx ... yyyy ... """ >>> B b'\nxxxx\nyyyy\n'
'xxx'
str
b'xxx' bytes bytes
str u'xxx'
U'xxx'
'xxx'
Conversions str
unicode str
bytes str
str.encode() bytes bytes.decode() str
bytes
bytes(S, encoding) str str(B, encoding) bytes Python 3.0 Strings in Action | 903
www.it-ebooks.info
encode
decode
>>> S = 'eggs' >>> S.encode() b'eggs' >>> bytes(S, encoding='ascii') b'eggs' >>> B = b'spam' >>> B.decode() 'spam' >>> str(B, encoding='ascii') 'spam'
sys str.encode
bytes bytes.decode str
bytes
str
str bytes
str B
S
>>> import sys >>> sys.platform 'win32' >>> sys.getdefaultencoding() 'utf-8' >>> bytes(S) TypeError: string argument without an encoding >>> str(B) "b'spam'" >>> len(str(B)) 7 >>> len(str(B, encoding='ascii')) 4
Coding Unicode Strings "\xNN"
904 | Chapter 36: Unicode and Byte Strings
"\uNNNN"
"\UNNNNNNNN"
www.it-ebooks.info
Coding ASCII Text
C:\misc> c:\python30\python >>> ord('X') 88 >>> chr(88) 'X' >>> S = 'XYZ' >>> S 'XYZ' >>> len(S) 3 >>> [ord(c) for c in S] [88, 89, 90]
>>> S.encode('ascii') b'XYZ' >>> S.encode('latin-1') b'XYZ' >>> S.encode('utf-8') b'XYZ'
bytes >>> S.encode('latin-1')[0] 88 >>> list(S.encode('latin-1')) [88, 89, 90]
Coding Non-ASCII Text 0xCD str
0xE8
str
Coding Unicode Strings | 905
www.it-ebooks.info
>>> chr(0xc4) 'Ä' >>> chr(0xe8) 'è' >>> S = '\xc4\xe8' >>> S 'Äè' >>> S = '\u00c4\u00e8' >>> S 'Äè' >>> len(S) 2
Encoding and Decoding Non-ASCII text
bytes >>> S = '\u00c4\u00e8' >>> S 'Äè' >>> len(S) 2 >>> S.encode('ascii') UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128) >>> S.encode('latin-1') b'\xc4\xe8' >>> S.encode('utf-8') b'\xc3\x84\xc3\xa8' >>> len(S.encode('latin-1')) 2 >>> len(S.encode('utf-8')) 4
open
>>> B = b'\xc4\xe8' >>> B b'\xc4\xe8'
906 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
>>> len(B) 2 >>> B.decode('latin-1') 'Äè' >>> B = b'\xc3\x84\xc3\xa8' >>> len(B) 4 >>> B.decode('utf-8') 'Äè' >>> len(B.decode('utf-8')) 2
Other Unicode Coding Techniques "\u..."
"\U...."
>>> S = 'A\u00c4B\U000000e8C' >>> S 'AÄBèC' >>> len(S) 5 >>> S.encode('latin-1') b'A\xc4B\xe8C' >>> len(S.encode('latin-1')) 5 >>> S.encode('utf-8') b'A\xc3\x84B\xc3\xa8C' >>> len(S.encode('utf-8')) 7
cp500
>>> S 'AÄBèC' >>> S.encode('cp500') b'\xc1c\xc2T\xc3' >>> S.encode('cp850') b'A\x8eB\x8aC' >>> S = 'spam' >>> S.encode('latin-1') b'spam' >>> S.encode('utf-8') b'spam' >>> S.encode('cp500')
Coding Unicode Strings | 907
www.it-ebooks.info
b'\xa2\x97\x81\x94' >>> S.encode('cp850') b'spam'
chr >>> S = 'A' + chr(0xC4) + 'B' + chr(0xE8) + 'C' >>> S 'AÄBèC'
str
bytes bytes
bytes
str
>>> S = 'A\xC4B\xE8C' >>> S 'AÄBèC' >>> S = 'A\u00C4B\U000000E8C' >>> S 'AÄBèC' >>> B = b'A\xC4B\xE8C' >>> B b'A\xc4B\xe8C' >>> B = b'A\u00C4B\U000000E8C' >>> B b'A\\u00C4B\\U000000E8C' >>> B = b'A\xC4B\xE8C' >>> B b'A\xc4B\xe8C' >>> print(B) b'A\xc4B\xe8C' >>> B.decode('latin-1') 'AÄBèC'
bytes str
>>> S = 'AÄBèC' >>> S 'AÄBèC' >>> B = b'AÄBèC' SyntaxError: bytes can only contain ASCII literal characters. >>> B = b'A\xC4B\xE8C' >>> B
908 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
b'A\xc4B\xe8C' >>> B.decode('latin-1') 'AÄBèC' >>> S.encode() b'A\xc3\x84B\xc3\xa8C' >>> S.encode('utf-8') b'A\xc3\x84B\xc3\xa8C' >>> B.decode() UnicodeDecodeError: 'utf8' codec can't decode bytes in position 1-2: ...
Converting Encodings
>>> S = 'AÄBèC' >>> S 'AÄBèC' >>> S.encode() b'A\xc3\x84B\xc3\xa8C' >>> T = S.encode('cp500') >>> T b'\xc1c\xc2T\xc3' >>> U = T.decode('cp500') >>> U 'AÄBèC' >>> U.encode() b'A\xc3\x84B\xc3\xa8C'
open
Coding Unicode Strings | 909
www.it-ebooks.info
Coding Unicode Strings in Python 2.6 unicode str str
bytes
C:\misc> c:\python26\python >>> import sys >>> sys.version '2.6 (r26:66721, Oct 2 2008, 11:35:03) [MSC v.1500 32 bit (Intel)]' >>> S = 'A\xC4B\xE8C' >>> print S AÄBèC >>> S.decode('latin-1') u'A\xc4B\xe8C' >>> S.decode('utf-8') UnicodeDecodeError: 'utf8' codec can't decode bytes in position 1-2: invalid data >>> S.decode('ascii') UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 1: ordinal not in range(128)
unicode
u'xxx'
>>> U = u'A\xC4B\xE8C' >>> U u'A\xc4B\xe8C' >>> print U AÄBèC
str
bytes
>>> U.encode('latin-1') 'A\xc4B\xe8C' >>> U.encode('utf-8') 'A\xc3\x84B\xc3\xa8C'
bytes unicode C:\misc> c:\python26\python >>> U = u'A\xC4B\xE8C' >>> U u'A\xc4B\xe8C'
910 | Chapter 36: Unicode and Byte Strings
"\u..." str
"\U..."
www.it-ebooks.info
>>> print U AÄBèC >>> U = u'A\u00C4B\U000000E8C' >>> U u'A\xc4B\xe8C' >>> print U AÄBèC >>> S = 'A\xC4B\xE8C' >>> S 'A\xc4B\xe8C' >>> print S A-BFC >>> print S.decode('latin-1') AÄBèC >>> S = 'A\u00C4B\U000000E8C' >>> S 'A\\u00C4B\\U000000E8C' >>> print S A\u00C4B\U000000E8C >>> len(S) 19
str
bytes
unicode
str unicode
str unicode str unicode
str str
unicode bytes
>>> u'ab' + 'cd' u'abcd'
re str
unicode
>>> str(u'spam') 'spam' >>> unicode('spam') u'spam'
unicode >>> S = 'A\xC4B\xE8C' >>> U = u'A\xC4B\xE8C' >>> S + U UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 1: ordinal not in range(128)
Coding Unicode Strings | 911
www.it-ebooks.info
>>> S.decode('latin-1') + U u'A\xc4B\xe8CA\xc4B\xe8C' >>> print S.decode('latin-1') + U AÄBèCAÄBèC
open str codecs.open open
unicode
Source File Character Set Encoding Declarations
# -*- coding: latin-1 -*-
# -*- coding: latin-1 -*-
myStr1 = 'aÄBèC' myStr2 = 'A\u00c4B\U000000e8C' myStr3 = 'A' + chr(0xC4) + 'B' + chr(0xE8) + 'C' import sys print('Default encoding:', sys.getdefaultencoding()) for aStr in myStr1, myStr2, myStr3:
912 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
print('{0}, strlen={1}, '.format(aStr, len(aStr)), end='') bytes1 = aStr.encode() bytes2 = aStr.encode('latin-1') #bytes3 = aStr.encode('ascii') print('byteslen1={0}, byteslen2={1}'.format(len(bytes1), len(bytes2)))
C:\misc> c:\python30\python text.py Default encoding: utf-8 aÄBèC, strlen=5, byteslen1=7, byteslen2=5 AÄBèC, strlen=5, byteslen1=7, byteslen2=5 AÄBèC, strlen=5, byteslen1=7, byteslen2=5
Using 3.0 Bytes Objects str
bytes bytes
str mat str
str
bytes
for
%
bytes str bytes
Method Calls str
bytes
dir __mod__
__rmod__
%
C:\misc> c:\python30\python
>>> set(dir('abc')) - set(dir(b'abc')) {'isprintable', 'format', '__mod__', 'encode', 'isidentifier', '_formatter_field_name_split', 'isnumeric', '__rmod__', 'isdecimal',
Using 3.0 Bytes Objects | 913
www.it-ebooks.info
'_formatter_parser', 'maketrans'}
>>> set(dir(b'abc')) - set(dir('abc')) {'decode', 'fromhex'}
str
bytes decode
bytes
str
encode
bytes bytes
bytes bytes str
>>> B = b'spam' >>> B.find(b'pa') 1 >>> B.replace(b'pa', b'XY') b'sXYm' >>> B.split(b'pa') [b's', b'm'] >>> B b'spam' >>> B[0] = 'x' TypeError: 'bytes' object does not support item assignment
str bytes >>> b'%s' % 99 TypeError: unsupported operand type(s) for %: 'bytes' and 'int' >>> '%s' % 99 '99' >>> b'{0}'.format(99) AttributeError: 'bytes' object has no attribute 'format' >>> '{0}'.format(99) '99'
Sequence Operations str
914 | Chapter 36: Unicode and Byte Strings
bytes
www.it-ebooks.info
bytes
bytes chr
>>> B = b'spam' >>> B b'spam' >>> B[0] 115 >>> B[-1] 109 >>> chr(B[0]) 's' >>> list(B) [115, 112, 97, 109] >>> B[1:], B[:-1] (b'pam', b'spa') >>> len(B) 4 >>> B + b'lmn' b'spamlmn' >>> B * 4 b'spamspamspamspam'
Other Ways to Make bytes Objects bytes bytes
b'...' str
bytes str str bytes
>>> B = b'abc' >>> B b'abc' >>> B = bytes('abc', 'ascii') >>> B b'abc' >>> ord('a') 97 >>> B = bytes([97, 98, 99])
Using 3.0 Bytes Objects | 915
www.it-ebooks.info
>>> B b'abc' >>> B = 'spam'.encode() >>> B b'spam' >>> >>> S = B.decode() >>> S 'spam'
str
bytes
Mixing String Types replace bytes
str str
unicode
>>> B = b'spam' >>> B.replace('pa', 'XY') TypeError: expected an object with the buffer interface >>> B.replace(b'pa', b'XY') b'sXYm' >>> B = B'spam' >>> B.replace(bytes('pa'), bytes('xy')) TypeError: string argument without an encoding >>> B.replace(bytes('pa', 'ascii'), bytes('xy', 'utf-8')) b'sxym'
>>> b'ab' + 'cd' TypeError: can't concat bytes to str >>> b'ab'.decode() + 'cd' 'abcd' >>> b'ab' + 'cd'.encode() b'abcd'
916 | Chapter 36: Unicode and Byte Strings
str
www.it-ebooks.info
>>> b'ab' + bytes('cd', 'ascii') b'abcd'
bytes bytes
Using 3.0 (and 2.6) bytearray Objects str
bytes
str
unicode bytearray bytes bytes bytearray
bytearray
bytearray
>>> S = 'spam' >>> C = bytearray(S) >>> C bytearray(b'spam')
>>> S = 'spam' >>> C = bytearray(S) TypeError: string argument without an encoding >>> C = bytearray(S, 'latin1') >>> C bytearray(b'spam') >>> B = b'spam' >>> C = bytearray(B) >>> C bytearray(b'spam')
bytearray
bytes
Using 3.0 (and 2.6) bytearray Objects | 917
www.it-ebooks.info
>>> C[0] 115 >>> C[0] = 'x' TypeError: an integer is required >>> C[0] = b'x' TypeError: an integer is required >>> C[0] = ord('x') >>> C bytearray(b'xpam') >>> C[1] = b'Y'[0] >>> C bytearray(b'xYam')
bytearray __iadd__ bytearray
__setitem__
+=
>>> set(dir(b'abc')) - set(dir(bytearray(b'abc'))) {'__getnewargs__'} >>> set(dir(bytearray(b'abc'))) - set(dir(b'abc')) {'insert', '__alloc__', 'reverse', 'extend', '__delitem__', 'pop', '__setitem__' , '__iadd__', 'remove', 'append', '__imul__'}
bytearray list(str)
>>> C bytearray(b'xYam') >>> C.append(b'LMN') TypeError: an integer is required >>> C.append(ord('L')) >>> C bytearray(b'xYamL') >>> C.extend(b'MNO') >>> C bytearray(b'xYamLMNO')
918 | Chapter 36: Unicode and Byte Strings
''.join(list)
www.it-ebooks.info
bytearray bytes
bytes
str
>>> C + b'!#' bytearray(b'xYamLMNO!#') >>> C[0] 120 >>> C[1:] bytearray(b'YamLMNO') >>> len(C) 8 >>> C bytearray(b'xYamLMNO') >>> C.replace('xY', 'sp') TypeError: Type str doesn't support the buffer API >>> C.replace(b'xY', b'sp') bytearray(b'spamLMNO') >>> C bytearray(b'xYamLMNO') >>> C * 4 bytearray(b'xYamLMNOxYamLMNOxYamLMNOxYamLMNO')
bytes bytearray
int
str
>>> B b'spam' >>> list(B) [115, 112, 97, 109] >>> C bytearray(b'xYamLMNO') >>> list(C) [120, 89, 97, 109, 76, 77, 78, 79] >>> S 'spam' >>> list(S) ['s', 'p', 'a', 'm']
Using 3.0 (and 2.6) bytearray Objects | 919
www.it-ebooks.info
str bytes bytearray
Using Text and Binary Files str
bytes
open '\n'
open "rb"
"rt"
"r"
open str bytes bytearray
Text File Basics
920 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
file C:\misc> c:\python30\python
>>> file = open('temp', 'w') >>> size = file.write('abc\n') >>> file.close() >>> file = open('temp') >>> text = file.read() >>> text 'abc\n' >>> print(text) abc
Text and Binary Modes in 3.0 str \n
\r\n
C:\misc> c:\python26\python >>> open('temp', 'w').write('abd\n') >>> open('temp', 'r').read() 'abd\n' >>> open('temp', 'rb').read() 'abd\r\n' >>> open('temp', 'wb').write('abc\n') >>> open('temp', 'r').read() 'abc\n' >>> open('temp', 'rb').read() 'abc\n'
str bytes str str
bytes
open
C:\misc> c:\python30\python
>>> open('temp', 'w').write('abc\n') 4 >>> open('temp', 'r').read() 'abc\n'
Using Text and Binary Files | 921
www.it-ebooks.info
>>> open('temp', 'rb').read() b'abc\r\n'
\n \r\n
\r\n \n open bytes
str
bytes
>>> open('temp', 'wb').write(b'abc\n') 4 >>> open('temp', 'r').read() 'abc\n' >>> open('temp', 'rb').read() b'abc\n'
\n
\r\n
"\x00"
>>> open('temp', 'wb').write(b'a\x00c') 3 >>> open('temp', 'r').read() 'a\x00c' >>> open('temp', 'rb').read() b'a\x00c'
bytes bytearray
bytes bytearray
bytes bytearray
>>> BA = bytearray(b'\x01\x02\x03') >>> open('temp', 'wb').write(BA) 3 >>> open('temp', 'r').read() '\x01\x02\x03'
922 | Chapter 36: Unicode and Byte Strings
bytes
www.it-ebooks.info
>>> open('temp', 'rb').read() b'\x01\x02\x03'
Type and Content Mismatches str bytes bytes
str
>>> open('temp', 'w').write('abc\n') 4 >>> open('temp', 'w').write(b'abc\n') TypeError: can't write bytes to text stream >>> open('temp', 'wb').write(b'abc\n') 4 >>> open('temp', 'wb').write('abc\n') TypeError: can't write str to binary stream
str
bytes str
bytes
str
str
bytes
bytes
>>> chr(0xFF) 'ÿ' >>> chr(0xFE) UnicodeEncodeError: 'charmap' codec can't encode character '\xfe' in position 1... >>> open('temp', 'w').write(b'\xFF\xFE\xFD') TypeError: can't write bytes to text stream >>> open('temp', 'w').write('\xFF\xFE\xFD') 3 >>> open('temp', 'wb').write(b'\xFF\xFE\xFD') 3 >>> open('temp', 'rb').read()
Using Text and Binary Files | 923
www.it-ebooks.info
b'\xff\xfe\xfd' >>> open('temp', 'r').read() UnicodeEncodeError: 'charmap' codec can't encode characters in position 2-3: ...
Using Unicode Files open
Reading and Writing Unicode in 3.0
C:\misc> c:\python30\python >>> S = 'A\xc4B\xe8C' >>> S 'AÄBèC' >>> len(S) 5
Manual encoding
>>> L = S.encode('latin-1') >>> L b'A\xc4B\xe8C' >>> len(L) 5 >>> U = S.encode('utf-8') >>> U b'A\xc3\x84B\xc3\xa8C' >>> len(U) 7
924 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
File output encoding open
>>> open('latindata', 'w', encoding='latin-1').write(S) 5 >>> open('utf8data', 'w', encoding='utf-8').write(S) 5 >>> open('latindata', 'rb').read() b'A\xc4B\xe8C' >>> open('utf8data', 'rb').read() b'A\xc3\x84B\xc3\xa8C'
File input decoding open
>>> open('latindata', 'r', encoding='latin-1').read() 'AÄBèC' >>> open('utf8data', 'r', encoding='utf-8').read() 'AÄBèC' >>> X = open('latindata', 'rb').read() >>> X.decode('latin-1') 'AÄBèC' >>> X = open('utf8data', 'rb').read() >>> X.decode() 'AÄBèC'
Decoding mismatches
str
>>> file = open('python.exe', 'r') >>> text = file.read() UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 2: ... >>> file = open('python.exe', 'rb')
Using Unicode Files | 925
www.it-ebooks.info
>>> data = file.read() >>> data[:20] b'MZ\x90\x00\x03\x00\x00\x00\x04\x00\x00\x00\xff\xff\x00\x00\xb8\x00\x00\x00'
\r\n \n open
Handling the BOM in 3.0
c:\misc> C:\Python30\python >>> import sys >>> sys.getdefaultencoding() 'utf-8' >>> open('spam.txt', 'rb').read() b'spam\r\nSPAM\r\n' >>> open('spam.txt', 'r').read() 'spam\nSPAM\n' >>> open('spam.txt', 'r', encoding='utf-8').read() 'spam\nSPAM\n'
>>> open('spam.txt', 'rb').read() b'\xef\xbb\xbfspam\r\nSPAM\r\n' >>> open('spam.txt', 'r').read() 'spam\nSPAM\n' >>> open('spam.txt', 'r', encoding='utf-8').read() '\ufeffspam\nSPAM\n' >>> open('spam.txt', 'r', encoding='utf-8-sig').read() 'spam\nSPAM\n'
926 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
>>> open('spam.txt', 'rb').read() b'\xfe\xff\x00s\x00p\x00a\x00m\x00\r\x00\n\x00S\x00P\x00A\x00M\x00\r\x00\n' >>> open('spam.txt', 'r').read() UnicodeEncodeError: 'charmap' codec can't encode character '\xfe' in position 1:... >>> open('spam.txt', 'r', encoding='utf-16').read() 'spam\nSPAM\n' >>> open('spam.txt', 'r', encoding='utf-16-be').read() '\ufeffspam\nSPAM\n'
>>> open('temp.txt', 'w', encoding='utf-8').write('spam\nSPAM\n') 10 >>> open('temp.txt', 'rb').read() b'spam\r\nSPAM\r\n' >>> open('temp.txt', 'w', encoding='utf-8-sig').write('spam\nSPAM\n') 10 >>> open('temp.txt', 'rb').read() b'\xef\xbb\xbfspam\r\nSPAM\r\n' >>> open('temp.txt', 'r').read() 'spam\nSPAM\n' >>> open('temp.txt', 'r', encoding='utf-8').read() '\ufeffspam\nSPAM\n' >>> open('temp.txt', 'r', encoding='utf-8-sig').read() 'spam\nSPAM\n'
>>> open('temp.txt', 10 >>> open('temp.txt', b'spam\r\nSPAM\r\n' >>> open('temp.txt', 'spam\nSPAM\n' >>> open('temp.txt', 'spam\nSPAM\n' >>> open('temp.txt', 'spam\nSPAM\n'
'w').write('spam\nSPAM\n') 'rb').read() 'r').read() 'r', encoding='utf-8').read() 'r', encoding='utf-8-sig').read()
Using Unicode Files | 927
www.it-ebooks.info
>>> sys.byteorder 'little' >>> open('temp.txt', 'w', encoding='utf-16').write('spam\nSPAM\n') 10 >>> open('temp.txt', 'rb').read() b'\xff\xfes\x00p\x00a\x00m\x00\r\x00\n\x00S\x00P\x00A\x00M\x00\r\x00\n\x00' >>> open('temp.txt', 'r', encoding='utf-16').read() 'spam\nSPAM\n' >>> open('temp.txt', 'w', encoding='utf-16-be').write('\ufeffspam\nSPAM\n') 11 >>> open('spam.txt', 'rb').read() b'\xfe\xff\x00s\x00p\x00a\x00m\x00\r\x00\n\x00S\x00P\x00A\x00M\x00\r\x00\n' >>> open('temp.txt', 'r', encoding='utf-16').read() 'spam\nSPAM\n' >>> open('temp.txt', 'r', encoding='utf-16-be').read() '\ufeffspam\nSPAM\n'
>>> open('temp.txt', 'w', encoding='utf-16-le').write('SPAM') 4 >>> open('temp.txt', 'rb').read() b'S\x00P\x00A\x00M\x00' >>> open('temp.txt', 'r', encoding='utf-16-le').read() 'SPAM' >>> open('temp.txt', 'r', encoding='utf-16').read() UnicodeError: UTF-16 stream does not start with BOM
Unicode Files in 2.6 str unicode
open
codecs.open
C:\misc> c:\python26\python >>> S = u'A\xc4B\xe8C' >>> print S AÄBèC >>> len(S) 5 >>> S.encode('latin-1') 'A\xc4B\xe8C' >>> S.encode('utf-8') 'A\xc3\x84B\xc3\xa8C' >>> import codecs
928 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
>>> codecs.open('latindata', 'w', encoding='latin-1').write(S) >>> codecs.open('utfdata', 'w', encoding='utf-8').write(S) >>> open('latindata', 'rb').read() 'A\xc4B\xe8C' >>> open('utfdata', 'rb').read() 'A\xc3\x84B\xc3\xa8C' >>> codecs.open('latindata', 'r', encoding='latin-1').read() u'A\xc4B\xe8C' >>> codecs.open('utfdata', 'r', encoding='utf-8').read() u'A\xc4B\xe8C'
Other String Tool Changes in 3.0 str bytes re struct
pickle
xml
The re Pattern Matching Module re find split
replace
re str bytes
(.*) ()
bytearray
.
* group
groups
C:\misc> c:\python30\python >>> import re >>> S = 'Bugger all down here on earth!' >>> B = b'Bugger all down here on earth!' >>> re.match('(.*) down (.*) on (.*)', S).groups() ('Bugger all', 'here', 'earth!') >>> re.match(b'(.*) down (.*) on (.*)', B).groups() (b'Bugger all', b'here', b'earth!')
unicode str
Other String Tool Changes in 3.0 | 929
www.it-ebooks.info
C:\misc> c:\python26\python >>> import re >>> S = 'Bugger all down here on earth!' >>> U = u'Bugger all down here on earth!' >>> re.match('(.*) down (.*) on (.*)', S).groups() ('Bugger all', 'here', 'earth!') >>> re.match('(.*) down (.*) on (.*)', U).groups() (u'Bugger all', u'here', u'earth!')
bytes
str str
bytes
C:\misc> c:\python30\python >>> import re >>> S = 'Bugger all down here on earth!' >>> B = b'Bugger all down here on earth!' >>> re.match('(.*) down (.*) on (.*)', B).groups() TypeError: can't use a string pattern on a bytes-like object >>> re.match(b'(.*) down (.*) on (.*)', S).groups() TypeError: can't use a bytes pattern on a string-like object >>> re.match(b'(.*) down (.*) on (.*)', bytearray(B)).groups() (bytearray(b'Bugger all'), bytearray(b'here'), bytearray(b'earth!')) >>> re.match('(.*) down (.*) on (.*)', bytearray(B)).groups() TypeError: can't use a string pattern on a bytes-like object
The struct Binary Data Module struct bytes bytearray
str
C:\misc> c:\python30\python >>> from struct import pack >>> pack('>i4sh', 7, 'spam', 8) b'\x00\x00\x00\x07spam\x00\x08' C:\misc> c:\python26\python >>> from struct import pack >>> pack('>i4sh', 7, 'spam', 8) '\x00\x00\x00\x07spam\x00\x08'
930 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
bytes
str bytes
C:\misc> c:\python30\python >>> import struct >>> B = struct.pack('>i4sh', 7, 'spam', 8) >>> B b'\x00\x00\x00\x07spam\x00\x08' >>> vals = struct.unpack('>i4sh', B) >>> vals (7, b'spam', 8) >>> vals = struct.unpack('>i4sh', B.decode()) TypeError: 'str' does not have the buffer interface
bytes C:\misc> c:\python30\python
>>> F = open('data.bin', 'wb') >>> import struct >>> data = struct.pack('>i4sh', 7, 'spam', 8) >>> data b'\x00\x00\x00\x07spam\x00\x08' >>> F.write(data) 10 >>> F.close()
>>> F = open('data.bin', 'rb') >>> data = F.read() >>> data b'\x00\x00\x00\x07spam\x00\x08' >>> values = struct.unpack('>i4sh', data) >>> values (7, b'spam', 8)
>>> values (7, b'spam', 8)
Other String Tool Changes in 3.0 | 931
www.it-ebooks.info
>>> bin(values[0]) '0b111' >>> values[0] & 0x01 1 >>> values[0] | 0b1010 15 >>> bin(values[0] | 0b1010) '0b1111' >>> bin(values[0] ^ 0b1010) '0b1101' >>> bool(values[0] & 0b100) True >>> bool(values[0] & 0b1000) False
bytes
>>> values[1] b'spam' >>> values[1][0] 115 >>> values[1][1:] b'pam' >>> bin(values[1][0]) '0b1110011' >>> bin(values[1][0] | 0b1100) '0b1111111' >>> values[1][0] | 0b1100 127
The pickle Object Serialization Module pickle shelve
pickle pickle dumps
C:\misc> C:\Python30\python >>> import pickle >>> pickle.dumps([1, 2, 3])
932 | Chapter 36: Unicode and Byte Strings
bytes
www.it-ebooks.info
b'\x80\x03]q\x00(K\x01K\x02K\x03e.' >>> pickle.dumps([1, 2, 3], protocol=0) b'(lp0\nL1L\naL2L\naL3L\na.'
str
bytes
dump >>> pickle.dump([1, 2, 3], open('temp', 'w')) TypeError: can't write bytes to text stream >>> pickle.dump([1, 2, 3], open('temp', 'w'), protocol=0) TypeError: can't write bytes to text stream >>> pickle.dump([1, 2, 3], open('temp', 'wb')) >>> open('temp', 'r').read() UnicodeEncodeError: 'charmap' codec can't encode character '\u20ac' in ...
>>> pickle.dump([1, 2, 3], open('temp', 'wb')) >>> pickle.load(open('temp', 'rb')) [1, 2, 3] >>> open('temp', 'rb').read() b'\x80\x03]q\x00(K\x01K\x02K\x03e.'
C:\misc> c:\python26\python >>> import pickle >>> pickle.dumps([1, 2, 3]) '(lp0\nI1\naI2\naI3\na.' >>> pickle.dumps([1, 2, 3], protocol=1) ']q\x00(K\x01K\x02K\x03e.' >>> pickle.dump([1, 2, 3], open('temp', 'w')) >>> pickle.load(open('temp')) [1, 2, 3] >>> open('temp').read() '(lp0\nI1\naI2\naI3\na.'
>>> >>> >>> [1,
import pickle pickle.dump([1, 2, 3], open('temp', 'wb')) pickle.load(open('temp', 'rb')) 2, 3]
Other String Tool Changes in 3.0 | 933
www.it-ebooks.info
XML Parsing Tools re
unicode str
str
2009 Learning Python Programming Python Python Pocket Reference O'Reilly Media
title Learning Python Programming Python Python Pocket Reference
re match search
934 | Chapter 36: Unicode and Byte Strings
findall
www.it-ebooks.info
import re text = open('mybooks.xml').read() found = re.findall('(.*)', text) for title in found: print(title)
from xml.dom.minidom import parse, Node xmltree = parse('mybooks.xml') for node1 in xmltree.getElementsByTagName('title'): for node2 in node1.childNodes: if node2.nodeType == Node.TEXT_NODE: print(node2.data)
import xml.sax.handler class BookHandler(xml.sax.handler.ContentHandler): def __init__(self): self.inTitle = False def startElement(self, name, attributes): if name == 'title': self.inTitle = True def characters(self, data): if self.inTitle: print(data) def endElement(self, name): if name == 'title': self.inTitle = False import xml.sax parser = xml.sax.make_parser() handler = BookHandler() parser.setContentHandler(handler) parser.parse('mybooks.xml')
etree
Other String Tool Changes in 3.0 | 935
www.it-ebooks.info
from xml.etree.ElementTree import parse tree = parse('mybooks.xml') for E in tree.findall('title'): print(E.text)
C:\misc> c:\python26\python domparse.py Learning Python Programming Python Python Pocket Reference C:\misc> c:\python30\python domparse.py Learning Python Programming Python Python Pocket Reference
unicode str C:\misc> c:\python30\python >>> from xml.dom.minidom import parse, Node >>> xmltree = parse('mybooks.xml') >>> for node in xmltree.getElementsByTagName('title'): ... for node2 in node.childNodes: ... if node2.nodeType == Node.TEXT_NODE: ... node2.data ... 'Learning Python' 'Programming Python' 'Python Pocket Reference' C:\misc> c:\python26\python >>> ...same code... ... u'Learning Python' u'Programming Python' u'Python Pocket Reference'
unicode
str
re struct pickle
936 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
Chapter Summary
bytes
Test Your Knowledge: Quiz
Test Your Knowledge: Answers str
bytes bytearray
bytes
str
Test Your Knowledge: Answers | 937
www.it-ebooks.info
str str
unicode unicode
bytearray
str
bytes bytearray
str
str
str unicode
str bytearray str bytes \xNN \uNNNN \UNNNNNNNN
str bytes bytearray \n
open codecs.open()
open
codecs.open()
938 | Chapter 36: Unicode and Byte Strings
www.it-ebooks.info
str re struct pickle str unicode
codecs.open()
bytes str bytes
xml
str open
Test Your Knowledge: Answers | 939
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 37
Managed Attributes
Why Manage Attributes? name person.name person.name = value
name
valid
transform
class Person: def getName(self): if not valid(): raise TypeError('cannot fetch name') else: return self.name.transform()
941
www.it-ebooks.info
def setName(self, value): if not valid(value): raise TypeError('cannot change name') else: self.name = transform(value) person = Person() person.getName() person.setName('value')
Inserting Code to Run on Attribute Access
__getattr__
__setattr__
__getattribute__ property
942 | Chapter 37: Managed Attributes
www.it-ebooks.info
CardHolder
Properties
property
self
The Basics attribute = property(fget, fset, fdel, doc)
None fget fdel
fset doc
fget fset
fdel
None fget None
Properties | 943
www.it-ebooks.info
A First Example name
_name
class Person: def __init__(self, name): self._name = name def getName(self): print('fetch...') return self._name def setName(self, value): print('change...') self._name = value def delName(self): print('remove...') del self._name name = property(getName, setName, delName, "name property docs") bob = Person('Bob Smith') print(bob.name) bob.name = 'Robert Smith' print(bob.name) del bob.name print('-'*20) sue = Person('Sue Jones') print(sue.name) print(Person.name.__doc__)
object object
fetch... Bob Smith change... fetch... Robert Smith remove... -------------------fetch... Sue Jones name property docs
944 | Chapter 37: Managed Attributes
www.it-ebooks.info
class Super: ...the original Person class code... name = property(getName, setName, delName, 'name property docs') class Person(Super): pass bob = Person('Bob Smith') ...rest unchanged...
Person Person
bob
name
Super
self
Computed Attributes
class PropSquare: def __init__(self, start): self.value = start def getX(self): return self.value ** 2 def setX(self, value): self.value = value X = property(getX, setX) P = PropSquare(3) Q = PropSquare(32) print(P.X) P.X = 4 print(P.X) print(Q.X)
X
9 16 1024
self
Properties | 945
www.it-ebooks.info
Coding Properties with Decorators @decorator def func(args): ...
decorator def func(args): ... func = decorator(func)
property class Person: @property def name(self): ...
property class Person: def name(self): ... name = property(name)
getter setter
getter class Person: def __init__(self, name): self._name = name @property def name(self): "name property docs" print('fetch...') return self._name @name.setter def name(self, value): print('change...') self._name = value @name.deleter def name(self): print('remove...') del self._name
946 | Chapter 37: Managed Attributes
deleter
www.it-ebooks.info
bob = Person('Bob Smith') print(bob.name) bob.name = 'Robert Smith' print(bob.name) del bob.name print('-'*20) sue = Person('Sue Jones') print(sue.name) print(Person.name.__doc__)
fetch... Bob Smith change... fetch... Robert Smith remove... -------------------fetch... Sue Jones name property docs
property
Descriptors property
self
Descriptors | 947
www.it-ebooks.info
The Basics
class Descriptor: "docstring goes here" def __get__(self, instance, owner): ... def __set__(self, instance, value): ... def __delete__(self, instance): ...
__set__ __set__
Descriptor method arguments
self instance __get__
self
948 | Chapter 37: Managed Attributes
owner instance instance.attr None class.attr
www.it-ebooks.info
X.attr __get__
Descriptor
Subject.attr object
>>> class Descriptor(object): ... def __get__(self, instance, owner): ... print(self, instance, owner, sep='\n') ... >>> class Subject: ... attr = Descriptor() ... >>> X = Subject() >>> X.attr >>> Subject.attr None
__get__ X.attr Subject.attr X.attr
->
__get__
Descriptor.__get__(Subject.attr, X, Subject)
None
Read-only descriptors __set__ X.a
a >>> ... ... >>> ... ... >>> >>> get >>> get >>> >>> 99 >>>
X
C
class D: def __get__(*args): print('get') class C: a = D() X = C() X.a C.a X.a = 99 X.a list(X.__dict__.keys())
Descriptors | 949
www.it-ebooks.info
['a'] >>> Y = C() >>> Y.a get >>> C.a get
>>> class D: ... def __get__(*args): print('get') ... def __set__(*args): raise AttributeError('cannot set') ... >>> class C: ... a = D() ... >>> X = C() >>> X.a get >>> X.a = 99 AttributeError: cannot set
__delete__ __del__
__delete__ __delattr__
A First Example name
instance
object class Name: "name descriptor docs" def __get__(self, instance, owner): print('fetch...') return instance._name
950 | Chapter 37: Managed Attributes
www.it-ebooks.info
def __set__(self, instance, value): print('change...') instance._name = value def __delete__(self, instance): print('remove...') del instance._name class Person: def __init__(self, name): self._name = name name = Name() bob = Person('Bob Smith') print(bob.name) bob.name = 'Robert Smith' print(bob.name) del bob.name print('-'*20) sue = Person('Sue Jones') print(sue.name) print(Name.__doc__)
self __get__ self instance owner
Name Person Person
fetch... Bob Smith change... fetch... Robert Smith remove... -------------------fetch... Sue Jones name descriptor docs
Person
Descriptors | 951
www.it-ebooks.info
... class Super: def __init__(self, name): self._name = name name = Name() class Person(Super): pass ...
class Person: def __init__(self, name): self._name = name class Name: "name descriptor docs" def __get__(self, instance, owner): print('fetch...') return instance._name def __set__(self, instance, value): print('change...') instance._name = value def __delete__(self, instance): print('remove...') del instance._name name = Name()
Name
... print(Person.Name.__doc__)
Computed Attributes
class DescSquare: def __init__(self, start): self.value = start def __get__(self, instance, owner):
952 | Chapter 37: Managed Attributes
Person
www.it-ebooks.info
return self.value ** 2 def __set__(self, instance, value): self.value = value class Client1: X = DescSquare(3) class Client2: X = DescSquare(32) c1 = Client1() c2 = Client2() print(c1.X) c1.X = 4 print(c1.X) print(c2.X)
9 16 1024
Using State Information in Descriptors name
class DescState: def __init__(self, value): self.value = value def __get__(self, instance, owner): print('DescState get') return self.value * 10 def __set__(self, instance, value): print('DescState set') self.value = value
Descriptors | 953
www.it-ebooks.info
class CalcAttrs: X = DescState(2) Y = 3 def __init__(self): self.Z = 4 obj = CalcAttrs() print(obj.X, obj.Y, obj.Z) obj.X = 5 obj.Y = 6 obj.Z = 7 print(obj.X, obj.Y, obj.Z)
value X Y
Z
DescState get 20 3 4 DescState set DescState get 50 6 7
_Y class InstState: def __get__(self, instance, owner): print('InstState get') return instance._Y * 100 def __set__(self, instance, value): print('InstState set') instance._Y = value
class CalcAttrs: X = DescState(2) Y = InstState() def __init__(self): self._Y = 3 self.Z = 4 obj = CalcAttrs() print(obj.X, obj.Y, obj.Z) obj.X = 5 obj.Y = 6 obj.Z = 7 print(obj.X, obj.Y, obj.Z)
954 | Chapter 37: Managed Attributes
Y
Z X
www.it-ebooks.info
X
Y
X
_Y
DescState InstState 20 300 4 DescState InstState DescState InstState 50 600 7
get get set set get get
How Properties and Descriptors Relate property property class Property: def __init__(self, fget=None, fset=None, fdel=None, doc=None): self.fget = fget self.fset = fset self.fdel = fdel self.__doc__ = doc def __get__(self, instance, instancetype=None): if instance is None: return self if self.fget is None: raise AttributeError("can't get attribute") return self.fget(instance) def __set__(self, instance, value): if self.fset is None: raise AttributeError("can't set attribute") self.fset(instance, value) def __delete__(self, instance): if self.fdel is None: raise AttributeError("can't delete attribute") self.fdel(instance) class Person:
Descriptors | 955
www.it-ebooks.info
def getName(self): ... def setName(self, value): ... name = Property(getName, setName)
Property
Property
__get__
Person
Person getName
@
Property setter
deleter self
property __slots__
__getattr__ and __getattribute__ __getattr__
__getattribute__
__getattr__ __getattribute__
__setattr__
956 | Chapter 37: Managed Attributes
__delattr__
www.it-ebooks.info
self __getattr__
__getattribute__
__setattr__
__delattr__
The Basics __getattr__ __setattr__ __getattribute__
def def def def
__getattr__(self, name): __getattribute__(self, name): __setattr__(self, name, value): __delattr__(self, name):
self
name value
None class Catcher: def __getattr__(self, name): print('Get:', name) def __setattr__(self, name, value):
__getattr__ and __getattribute__ | 957
www.it-ebooks.info
print('Set:', name, value) X = Catcher() X.job X.pay X.pay = 99
class Wrapper: def __init__(self, object): self.wrapped = object def __getattr__(self, attrname): print('Trace:', attrname) return getattr(self.wrapped, attrname)
Avoiding loops in attribute interception methods __getattr__ __getattribute__
__setattr__
__getattribute__ __getattribute__ def __getattribute__(self, name): x = self.other
object def __getattribute__(self, name): x = object.__getattribute__(self, 'other')
__setattr__ __setattr__ def __setattr__(self, name, value): self.other = value
__dict__
958 | Chapter 37: Managed Attributes
www.it-ebooks.info
def __setattr__(self, name, value): self.__dict__['other'] = value
__setattr__ __getattribute__ def __setattr__(self, name, value): object.__setattr__(self, 'other', value)
__dict__ __getattribute__ def __getattribute__(self, name): x = self.__dict__['other']
__dict__
__getattribute__
__delattr__ __setattr__
A First Example
class Person: def __init__(self, name): self._name = name def __getattr__(self, attr): if attr == 'name': print('fetch...') return self._name else: raise AttributeError(attr) def __setattr__(self, attr, value): if attr == 'name': print('change...') attr = '_name' self.__dict__[attr] = value def __delattr__(self, attr): if attr == 'name': print('remove...') attr = '_name' del self.__dict__[attr]
__getattr__ and __getattribute__ | 959
www.it-ebooks.info
bob = Person('Bob Smith') print(bob.name) bob.name = 'Robert Smith' print(bob.name) del bob.name print('-'*20) sue = Person('Sue Jones') print(sue.name) #print(Person.name.__doc__)
__init__
__setattr__
fetch... Bob Smith change... fetch... Robert Smith remove... -------------------fetch... Sue Jones
__getattribute__
def __getattribute__(self, attr): if attr == 'name': print('fetch...') attr = '_name' return object.__getattribute__(self, attr)
__getattr__
__getattribute__
960 | Chapter 37: Managed Attributes
__getattr__
www.it-ebooks.info
Computed Attributes X class AttrSquare: def __init__(self, start): self.value = start def __getattr__(self, attr): if attr == 'X': return self.value ** 2 else: raise AttributeError(attr) def __setattr__(self, attr, value): if attr == 'X': attr = 'value' self.__dict__[attr] = value A = AttrSquare(3) B = AttrSquare(32) print(A.X) A.X = 4 print(A.X) print(B.X)
9 16 1024
__getattribute__ __getattribute__
__getattr__ __setattr__ __dict__ class AttrSquare: def __init__(self, start): self.value = start def __getattribute__(self, attr): if attr == 'X': return self.value ** 2 else: return object.__getattribute__(self, attr) def __setattr__(self, attr, value):
__getattr__ and __getattribute__ | 961
www.it-ebooks.info
if attr == 'X': attr = 'value' object.__setattr__(self, attr, value)
self.value=start self.value
__setattr__ __getattribute__
__getattribute__ __getattr__
__getattribute__ X value __getattribute__
value def __getattribute__(self, attr): if attr == 'X': return object.__getattribute__(self, 'value') ** 2
print
__getattr__ and __getattribute__ Compared __getattr__ attr2
attr3
class GetAttr: attr1 = 1 def __init__(self): self.attr2 = 2 def __getattr__(self, attr): print('get: ' + attr) return 3 X = GetAttr() print(X.attr1) print(X.attr2) print(X.attr3) print('-'*40) class GetAttribute(object): attr1 = 1 def __init__(self): self.attr2 = 2 def __getattribute__(self, attr): print('get: ' + attr) if attr == 'attr3': return 3
962 | Chapter 37: Managed Attributes
__getattribute__ attr1
www.it-ebooks.info
else: return object.__getattribute__(self, attr) X = GetAttribute() print(X.attr1) print(X.attr2) print(X.attr3)
__getattr__ __getattribute__
attr3
1 2 get: attr3 3 ---------------------------------------get: attr1 1 get: attr2 2 get: attr3 3
__getattribute__
__getattr__
Management Techniques Compared
square
cube
class Powers: def __init__(self, square, cube): self._square = square self._cube = cube def getSquare(self): return self._square ** 2 def setSquare(self, value): self._square = value square = property(getSquare, setSquare) def getCube(self): return self._cube ** 3 cube = property(getCube)
__getattr__ and __getattribute__ | 963
www.it-ebooks.info
X = Powers(3, 4) print(X.square) print(X.cube) X.square = 5 print(X.square)
class DescSquare: def __get__(self, instance, owner): return instance._square ** 2 def __set__(self, instance, value): instance._square = value class DescCube: def __get__(self, instance, owner): return instance._cube ** 3 class Powers: square = DescSquare() cube = DescCube() def __init__(self, square, cube): self._square = square self._cube = cube X = Powers(3, 4) print(X.square) print(X.cube) X.square = 5 print(X.square)
__getattr__ __setattrr__
class Powers: def __init__(self, square, cube): self._square = square self._cube = cube def __getattr__(self, name): if name == 'square': return self._square ** 2 elif name == 'cube': return self._cube ** 3
964 | Chapter 37: Managed Attributes
www.it-ebooks.info
else: raise TypeError('unknown attr:' + name) def __setattr__(self, name, value): if name == 'square': self.__dict__['_square'] = value else: self.__dict__[name] = value X = Powers(3, 4) print(X.square) print(X.cube) X.square = 5 print(X.square)
__getattribute__
class Powers: def __init__(self, square, cube): self._square = square self._cube = cube def __getattribute__(self, name): if name == 'square': return object.__getattribute__(self, '_square') ** 2 elif name == 'cube': return object.__getattribute__(self, '_cube') ** 3 else: return object.__getattribute__(self, name) def __setattr__(self, name, value): if name == 'square': self.__dict__['_square'] = value else: self.__dict__[name] = value X = Powers(3, 4) print(X.square) print(X.cube) X.square = 5 print(X.square)
9 64 25
__getattr__ and __getattribute__ | 965
www.it-ebooks.info
Intercepting Built-in Operation Attributes __getattr__
__getattribute__
__str__ __add__
__getitem__
+ __getattr__
__getattribute__
__getattr__ __getattribute__
__getattr__
__getattribute__
__getattr__ __getattribute__
966 | Chapter 37: Managed Attributes
www.it-ebooks.info
class GetAttr: eggs = 88 def __init__(self): self.spam = 77 def __len__(self): print('__len__: 42') return 42 def __getattr__(self, attr): print('getattr: ' + attr) if attr == '__str__': return lambda *args: '[Getattr str]' else: return lambda *args: None class GetAttribute(object): eggs = 88 def __init__(self): self.spam = 77 def __len__(self): print('__len__: 42') return 42 def __getattribute__(self, attr): print('getattribute: ' + attr) if attr == '__str__': return lambda *args: '[GetAttribute str]' else: return lambda *args: None for Class in GetAttr, GetAttribute: print('\n' + Class.__name__.ljust(50, '=')) X = Class() X.eggs X.spam X.other len(X) try:
X[0] except: print('fail []') try:
X + 99 except: print('fail +') try:
X() except: print('fail ()') X.__call__() print(X.__str__()) print(X)
__getattr__ and __getattribute__ | 967
www.it-ebooks.info
__getattr__ __getattribute__ C:\misc> c:\python26\python getattr.py GetAttr=========================================== getattr: other __len__: 42 getattr: __getitem__ getattr: __coerce__ getattr: __add__ getattr: __call__ getattr: __call__ getattr: __str__ [Getattr str] getattr: __str__ [Getattr str] GetAttribute====================================== getattribute: eggs getattribute: spam getattribute: other __len__: 42 fail [] fail + fail () getattribute: __call__ getattribute: __str__ [GetAttribute str]
__getattr__ __str__
__call__ __getattribute__
__getattribute__ object object __getattr__
C:\misc> c:\python30\python getattr.py GetAttr=========================================== getattr: other __len__: 42 fail [] fail + fail ()
968 | Chapter 37: Managed Attributes
www.it-ebooks.info
getattr: __call__ GetAttribute====================================== getattribute: eggs getattribute: spam getattribute: other __len__: 42 fail [] fail + fail () getattribute: __call__ getattribute: __str__ [GetAttribute str]
print __str__
__getattr__ object
__str__
__getattribute__
__call__ __str__ __call__
__getattr__
__len__ __getattr__ attribute__
__get
__len__
Private
__getattr__ and __getattribute__ | 969
www.it-ebooks.info
__getattribute__ __str__ __getattribute__
os.popen __getattr__ next(X) __next__ X.__next__() __str__ __call__ __getattr__
Delegation-Based Managers Revisited Manager
class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay def lastName(self): return self.name.split()[-1] def giveRaise(self, percent): self.pay = int(self.pay * (1 + percent)) def __str__(self): return '[Person: %s, %s]' % (self.name, self.pay) class Manager: def __init__(self, name, pay): self.person = Person(name, 'mgr', pay) def giveRaise(self, percent, bonus=.10): self.person.giveRaise(percent + bonus) def __getattr__(self, attr): return getattr(self.person, attr)
970 | Chapter 37: Managed Attributes
www.it-ebooks.info
def __str__(self): return str(self.person) if __name__ == '__main__': sue = Person('Sue Jones', job='dev', pay=100000) print(sue.lastName()) sue.giveRaise(.10) print(sue) tom = Manager('Tom Jones', 50000) print(tom.lastName()) tom.giveRaise(.10) print(tom)
lastName __getattr__
Manager Person Person
giveRaise
Manager
C:\misc> c:\python30\python getattr.py Jones [Person: Sue Jones, 110000] Jones [Person: Tom Jones, 60000]
Manager __str__
Person Manager.__str__
__str__
class Manager: def __init__(self, name, pay): self.person = Person(name, 'mgr', pay) def giveRaise(self, percent, bonus=.10): self.person.giveRaise(percent + bonus) def __getattr__(self, attr): return getattr(self.person, attr)
__getattr__ __str__ sue
Manager object Person
__str__
C:\misc> c:\python30\python person.py Jones [Person: Sue Jones, 110000] Jones
__str__
__getattr__
__str__ __getattr__ and __getattribute__ | 971
www.it-ebooks.info
C:\misc> c:\python26\python person.py Jones [Person: Sue Jones, 110000] Jones [Person: Tom Jones, 60000]
__getattribute__
__getattr__
class Manager: def __init__(self, name, pay): self.person = Person(name, 'mgr', pay) def giveRaise(self, percent, bonus=.10): self.person.giveRaise(percent + bonus) def __getattribute__(self, attr): print('**', attr) if attr in ['person', 'giveRaise']: return object.__getattribute__(self, attr) else: return getattr(self.person, attr)
__str__
Manager Person
C:\misc> c:\python30\python person.py Jones [Person: Sue Jones, 110000] ** lastName ** person Jones ** giveRaise ** person
__getattribute__ self.person self.person
__str__ __getattribute__
class Manager: def __init__(self, name, pay): self.person = Person(name, 'mgr', pay) def __getattribute__(self, attr): print('**', attr) person = object.__getattribute__(self, 'person') if attr == 'giveRaise': return lambda percent: person.giveRaise(percent+.10) else: return getattr(person, attr)
972 | Chapter 37: Managed Attributes
www.it-ebooks.info
def __str__(self): person = object.__getattribute__(self, 'person') return str(person)
__str__ Jones [Person: Sue Jones, 110000] ** lastName Jones ** giveRaise [Person: Tom Jones, 60000]
Manager __str__ __getattr__
Example: Attribute Validations CardHolder
Using Properties to Validate
Example: Attribute Validations | 973
www.it-ebooks.info
__init__ self.name
__name
setName __name
name name age
addr
acct remain
class CardHolder: acctlen = 8 retireage = 59.5 def __init__(self, acct, name, age, addr): self.acct = acct self.name = name self.age = age self.addr = addr def getName(self): return self.__name def setName(self, value): value = value.lower().replace(' ', '_') self.__name = value name = property(getName, setName) def getAge(self): return self.__age def setAge(self, value): if value < 0 or value > 150: raise ValueError('invalid age') else: self.__age = value age = property(getAge, setAge) def getAcct(self): return self.__acct[:-3] + '***' def setAcct(self, value): value = value.replace('-', '') if len(value) != self.acctlen: raise TypeError('invald acct number') else: self.__acct = value acct = property(getAcct, setAcct) def remainGet(self):
974 | Chapter 37: Managed Attributes
www.it-ebooks.info
return self.retireage - self.age remain = property(remainGet)
Self-test code
try bob = CardHolder('1234-5678', 'Bob Smith', 40, '123 main st') print(bob.acct, bob.name, bob.age, bob.remain, bob.addr, sep=' / ') bob.name = 'Bob Q. Smith' bob.age = 50 bob.acct = '23-45-67-89' print(bob.acct, bob.name, bob.age, bob.remain, bob.addr, sep=' / ') sue = CardHolder('5678-12-34', 'Sue Jones', 35, '124 main st') print(sue.acct, sue.name, sue.age, sue.remain, sue.addr, sep=' / ') try: sue.age = 200 except: print('Bad age for Sue') try:
sue.remain = 5 except: print("Can't set sue.remain") try:
sue.acct = '1234567' except: print('Bad acct for Sue')
12345*** / bob_smith / 40 / 19.5 / 123 main st 23456*** / bob_q._smith / 50 / 9.5 / 123 main st 56781*** / sue_jones / 35 / 24.5 / 124 main st Bad age for Sue Can't set sue.remain Bad acct for Sue
Using Descriptors to Validate
Example: Attribute Validations | 975
www.it-ebooks.info
__init__
__set__ self.name
Name.__set__() name name
CardHolder name name age
acct
addr remain remain
class CardHolder: acctlen = 8 retireage = 59.5 def __init__(self, acct, name, age, addr): self.acct = acct self.name = name self.age = age self.addr = addr class Name: def __get__(self, instance, owner): return self.name def __set__(self, instance, value): value = value.lower().replace(' ', '_') self.name = value name = Name() class Age: def __get__(self, instance, owner): return self.age def __set__(self, instance, value): if value < 0 or value > 150: raise ValueError('invalid age') else: self.age = value age = Age()
976 | Chapter 37: Managed Attributes
www.it-ebooks.info
class Acct: def __get__(self, instance, owner): return self.acct[:-3] + '***' def __set__(self, instance, value): value = value.replace('-', '') if len(value) != instance.acctlen: raise TypeError('invald acct number') else: self.acct = value acct = Acct() class Remain: def __get__(self, instance, owner): return instance.retireage - instance.age def __set__(self, instance, value): raise TypeError('cannot set remain') remain = Remain()
Using __getattr__ to Validate __getattr__
__getattr__
__setattr__ __init__ __setattr__
self.name
acct
__setattr__ name __getattr__ __getattr__
name acct
_acct name age
acct
addr remain
__getattr__
addr
Example: Attribute Validations | 977
www.it-ebooks.info
__getattr__ class CardHolder: acctlen = 8 retireage = 59.5 def __init__(self, acct, name, age, addr): self.acct = acct self.name = name self.age = age self.addr = addr def __getattr__(self, name): if name == 'acct': return self._acct[:-3] + '***' elif name == 'remain': return self.retireage - self.age else: raise AttributeError(name) def __setattr__(self, name, value): if name == 'name': value = value.lower().replace(' ', '_') elif name == 'age': if value < 0 or value > 150: raise ValueError('invalid age') elif name == 'acct': name = '_acct' value = value.replace('-', '') if len(value) != self.acctlen: raise TypeError('invald acct number') elif name == 'remain': raise TypeError('cannot set remain') self.__dict__[name] = value
Using __getattribute__ to Validate __getattribute__
__setattr__ __getattr__ __getattribute__ acct
978 | Chapter 37: Managed Attributes
acct
www.it-ebooks.info
addr
class CardHolder: acctlen = 8 retireage = 59.5 def __init__(self, acct, name, age, addr): self.acct = acct self.name = name self.age = age self.addr = addr def __getattribute__(self, name): superget = object.__getattribute__ if name == 'acct': return superget(self, 'acct')[:-3] + '***' elif name == 'remain': return superget(self, 'retireage') - superget(self, 'age') else: return superget(self, name) def __setattr__(self, name, value): if name == 'name': value = value.lower().replace(' ', '_') elif name == 'age': if value < 0 or value > 150: raise ValueError('invalid age') elif name == 'acct': value = value.replace('-', '') if len(value) != self.acctlen: raise TypeError('invald acct number') elif name == 'remain': raise TypeError('cannot set remain') self.__dict__[name] = value
Chapter Summary __getattr__
__getattribute__
Chapter Summary | 979
www.it-ebooks.info
Test Your Knowledge: Quiz __getattr__
__getattribute__
__getattr__
__getattri
bute__
Test Your Knowledge: Answers __getattr__ __getattribute__ __getattr__ __getattribute__
property
name = property(name) deleter
980 | Chapter 37: Managed Attributes
setter
www.it-ebooks.info
__getattr__
__getattribute__
__getattr__ __getattribute__ __setattr__ __getattr__ __getattribute__
An argument is a connected series of statements intended to establish a proposition. No it isn't. Yes it is! It's not just contradiction. Look, if I argue with you, I must take up a contrary position. Yes, but that's not just saying "No it isn't." Yes it is! No it isn't! Yes it is! No it isn't. Argument is an intellectual process. Contradiction is just the automatic gainsaying of any statement the other person makes. (short pause) No it isn't. It is. Not at all. Now look...
Test Your Knowledge: Quiz | 981
www.it-ebooks.info
www.it-ebooks.info
CHAPTER 38
Decorators
@ property
What’s a Decorator?
983
www.it-ebooks.info
def class
Managing Calls and Instances
def
Managing Functions and Classes
Using and Defining Decorators
984 | Chapter 38: Decorators
class
www.it-ebooks.info
Why Decorators?
What’s a Decorator? | 985
www.it-ebooks.info
The Basics
Function Decorators def
Usage def @
@decorator def F(arg): ... F(99)
decorator F def F(arg): ... F = decorator(F) F(99)
986 | Chapter 38: Decorators
www.it-ebooks.info
def F
func(6, 7) decorator(func)(6, 7)
class C: @staticmethod def meth(...): ... class C: @property def name(self): ...
def
Implementation
def decorator(F): return F @decorator def func(): ...
The Basics | 987
www.it-ebooks.info
def decorator(F):
@decorator def func(): ...
def decorator(F): def wrapper(*args): return wrapper @decorator def func(x, y): ... func(6, 7)
decorator
func wrapper
wrapper func
class decorator: def __init__(self, func): self.func = func def __call__(self, *args):
@decorator def func(x, y): ... func(6, 7)
func decorator
988 | Chapter 38: Decorators
__call__ __call__
www.it-ebooks.info
func
Supporting method decoration
class decorator: def __init__(self, func): self.func = func def __call__(self, *args): class C: @decorator def method(self, x, y):
self
__call__
decorator
C *args
def decorator(F): def wrapper(*args): return wrapper @decorator def func(x, y): ... func(6, 7) class C: @decorator def method(self, x, y): ... X = C() X.method(6, 7)
wrapper
C
self
The Basics | 989
www.it-ebooks.info
self
Class Decorators
Usage class decorator @decorator class C: ... x = C(99)
class C: ... C = decorator(C) x = C(99)
Implementation
990 | Chapter 38: Decorators
www.it-ebooks.info
def decorator(C): return C @decorator class C: ...
def decorator(C):
@decorator class C: ...
def decorator(cls): class Wrapper: def __init__(self, *args): self.wrapped = cls(*args) def __getattr__(self, name): return getattr(self.wrapped, name) return Wrapper @decorator class C: def __init__(self, x, y): self.attr = 'spam' x = C(6, 7) print(x.attr)
__getattr__
The Basics | 991
www.it-ebooks.info
__init__
__call__
Supporting multiple instances
class Decorator: def __init__(self, C): self.C = C def __call__(self, *args): self.wrapped = self.C(*args) return self def __getattr__(self, attrname): return getattr(self.wrapped, attrname) @Decorator class C: ... x = C() y = C()
Decorator __call__
def decorator(C): class Wrapper: def __init__(self, *args): self.wrapped = C(*args) return Wrapper class Wrapper: ... def decorator(C): def onCall(*args): return Wrapper(C(*args)) return onCall
992 | Chapter 38: Decorators
www.it-ebooks.info
Decorator Nesting
@A @B @C def f(...): ...
def f(...): ... f = A(B(C(f)))
@spam @eggs class C: ... X = C()
class C: ... C = spam(eggs(C)) X = C()
C spam
eggs
The Basics | 993
www.it-ebooks.info
def d1(F): return F def d2(F): return F def d3(F): return F @d1 @d2 @d3 def func(): print('spam') func()
def d1(F): return lambda: 'X' + F() def d2(F): return lambda: 'Y' + F() def d3(F): return lambda: 'Z' + F() @d1 @d2 @d3 def func(): return 'spam' print(func())
lambda
Decorator Arguments
@decorator(A, B) def F(arg): ... F(99)
decorator
994 | Chapter 38: Decorators
www.it-ebooks.info
def F(arg): ... F = decorator(A, B)(F) F(99)
def decorator(A, B): def actualDecorator(F): return callable return actualDecorator
Decorators Manage Functions and Classes, Too
def decorate(O): return O @decorator def F(): ... @decorator class C: ...
The Basics | 995
www.it-ebooks.info
Coding Function Decorators
Tracing Calls
class tracer: def __init__(self, func): self.calls = 0 self.func = func def __call__(self, *args): self.calls += 1 print('call %s to %s' % (self.calls, self.func.__name__)) self.func(*args) @tracer def spam(a, b, c): print(a + b + c)
*args
>>> from decorator1 import spam >>> spam(1, 2, 3) call 1 to spam 6 >>> spam('a', 'b', 'c') call 2 to spam abc >>> spam.calls 2
996 | Chapter 38: Decorators
www.it-ebooks.info
>>> spam
tracer spam tracer @
calls = 0 def tracer(func, *args): global calls calls += 1 print('call %s to %s' % (calls, func.__name__)) func(*args) def spam(a, b, c): print(a, b, c) >>> spam(1, 2, 3) 1 2 3 >>> tracer(spam, 1, 2, 3) call 1 to spam 1 2 3
@
State Information Retention Options
Coding Function Decorators | 997
www.it-ebooks.info
Class instance attributes
class tracer: def __init__(self, func): self.calls = 0 self.func = func def __call__(self, *args, **kwargs): self.calls += 1 print('call %s to %s' % (self.calls, self.func.__name__)) return self.func(*args, **kwargs) @tracer def spam(a, b, c): print(a + b + c) @tracer def eggs(x, y): print(x ** y) spam(1, 2, 3) spam(a=4, b=5, c=6) eggs(2, 16) eggs(4, y=4)
spam call 1 6 call 2 15 call 1 65536 call 2 256
eggs
to spam to spam to eggs to eggs
Enclosing scopes and globals def
998 | Chapter 38: Decorators
def
www.it-ebooks.info
calls = 0 def tracer(func): def wrapper(*args, **kwargs): global calls calls += 1 print('call %s to %s' % (calls, func.__name__)) return func(*args, **kwargs) return wrapper @tracer def spam(a, b, c): print(a + b + c) @tracer def eggs(x, y): print(x ** y) spam(1, 2, 3) spam(a=4, b=5, c=6) eggs(2, 16) eggs(4, y=4)
call 1 6 call 2 15 call 3 65536 call 4 256
to spam to spam to eggs to eggs
Enclosing scopes and nonlocals
nonlocal
def tracer(func): calls = 0 def wrapper(*args, **kwargs):
Coding Function Decorators | 999
www.it-ebooks.info
nonlocal calls calls += 1 print('call %s to %s' % (calls, func.__name__)) return func(*args, **kwargs) return wrapper @tracer def spam(a, b, c): print(a + b + c) @tracer def eggs(x, y): print(x ** y) spam(1, 2, 3) spam(a=4, b=5, c=6) eggs(2, 16) eggs(4, y=4)
call 1 6 call 2 15 call 1 65536 call 2 256
to spam to spam to eggs to eggs
Function attributes nonlocal
func.attr=value wrapper.calls
nonlocal
def tracer(func): def wrapper(*args, **kwargs): wrapper.calls += 1 print('call %s to %s' % (wrapper.calls, func.__name__)) return func(*args, **kwargs) wrapper.calls = 0 return wrapper
wrapper wrapper.calls
tracer wrapper
1000 | Chapter 38: Decorators
nonlocal
www.it-ebooks.info
nonlocal
Class Blunders I: Decorating Class Methods tracer self tracer
*args self *args
class tracer: def __init__(self, func): self.calls = 0 self.func = func def __call__(self, *args, **kwargs): self.calls += 1 print('call %s to %s' % (self.calls, self.func.__name__)) return self.func(*args, **kwargs)
@tracer def spam(a, b, c): print(a + b + c) spam(1, 2, 3) spam(a=4, b=5, c=6)
Coding Function Decorators | 1001
www.it-ebooks.info
Person class Person: def __init__(self, name, pay): self.name = name self.pay = pay @tracer def giveRaise(self, percent): self.pay *= (1.0 + percent) @tracer def lastName(self): return self.name.split()[-1] bob = Person('Bob Smith', 50000) bob.giveRaise(.25) print(bob.lastName())
self Person Person
tracer tracer self
tracer
__call__ Person
__call__
tracer
tracer
self tracer
Person
self
tracer
__call__ Person
(0.25,) {} call 1 to giveRaise Traceback (most recent call last): File "C:/misc/tracer.py", line 56, in bob.giveRaise(.25) File "C:/misc/tracer.py", line 9, in __call__ return self.func(*args, **kwargs) TypeError: giveRaise() takes exactly 2 positional arguments (1 given)
self
1002 | Chapter 38: Decorators
www.it-ebooks.info
Using nested functions to decorate methods
def self
Person *args
self
def tracer(func): calls = 0 def onCall(*args, **kwargs): nonlocal calls calls += 1 print('call %s to %s' % (calls, func.__name__)) return func(*args, **kwargs) return onCall
@tracer def spam(a, b, c): print(a + b + c) spam(1, 2, 3) spam(a=4, b=5, c=6)
class Person: def __init__(self, name, pay): self.name = name self.pay = pay @tracer def giveRaise(self, percent): self.pay *= (1.0 + percent) @tracer def lastName(self): return self.name.split()[-1] print('methods...') bob = Person('Bob Smith', 50000) sue = Person('Sue Jones', 100000) print(bob.name, sue.name)
Coding Function Decorators | 1003
www.it-ebooks.info
sue.giveRaise(.10) print(sue.pay) print(bob.lastName(), sue.lastName())
call 1 to spam 6 call 2 to spam 15 methods... Bob Smith Sue Jones call 1 to giveRaise 110000.0 call 1 to lastName call 2 to lastName Smith Jones
Using descriptors to decorate methods
__get__ object class Descriptor(object): def __get__(self, instance, owner): ... class Subject: attr = Descriptor() X = Subject() X.attr
__set__
__del__ __get__
class tracer(object): def __init__(self, func): self.calls = 0 self.func = func def __call__(self, *args, **kwargs): self.calls += 1 print('call %s to %s' % (self.calls, self.func.__name__)) return self.func(*args, **kwargs) def __get__(self, instance, owner): return wrapper(self, instance)
1004 | Chapter 38: Decorators
www.it-ebooks.info
class wrapper: def __init__(self, desc, subj): self.desc = desc self.subj = subj def __call__(self, *args, **kwargs): return self.desc(self.subj, *args, **kwargs) @tracer def spam(a, b, c): ...same as prior... class Person: @tracer def giveRaise(self, percent): ...same as prior...
__call__
__get__ instance.method
__call__
__get__
(args...)
sue.giveRaise(.10)
tracer.__get__
giveRaise
__call__ tracer.__call__
Person
wrapper
wrapper wrapper __call__ calls
class tracer(object): def __init__(self, func): self.calls = 0 self.func = func def __call__(self, *args, **kwargs): self.calls += 1 print('call %s to %s' % (self.calls, self.func.__name__)) return self.func(*args, **kwargs) def __get__(self, instance, owner): def wrapper(*args, **kwargs):
Coding Function Decorators | 1005
www.it-ebooks.info
return self(instance, *args, **kwargs) return wrapper
print
staticmethod
Timing Calls
map import time class timer: def __init__(self, func): self.func = func self.alltime = 0 def __call__(self, *args, **kargs): start = time.clock() result = self.func(*args, **kargs) elapsed = time.clock() - start self.alltime += elapsed print('%s: %.5f, %.5f' % (self.func.__name__, elapsed, self.alltime)) return result @timer def listcomp(N): return [x * 2 for x in range(N)] @timer def mapcall(N): return map((lambda x: x * 2), range(N))
1006 | Chapter 38: Decorators
www.it-ebooks.info
result = listcomp(5) listcomp(50000) listcomp(500000) listcomp(1000000) print(result) print('allTime = %s' % listcomp.alltime) print('') result = mapcall(5) mapcall(50000) mapcall(500000) mapcall(1000000) print(result) print('allTime = %s' % mapcall.alltime) print('map/comp = %s' % round(mapcall.alltime / listcomp.alltime, 3))
def
listcomp: listcomp: listcomp: listcomp: [0, 2, 4, allTime =
0.00002, 0.00002 0.00910, 0.00912 0.09105, 0.10017 0.17605, 0.27622 6, 8] 0.276223304917
mapcall: 0.00003, 0.00003 mapcall: 0.01363, 0.01366 mapcall: 0.13579, 0.14945 mapcall: 0.27648, 0.42593 [0, 2, 4, 6, 8] allTime = 0.425933533452 map/comp = 1.542
map map
map list(map()) map
map
Coding Function Decorators | 1007
www.it-ebooks.info
range ... import sys @timer def listcomp(N): return [x * 2 for x in range(N)] if sys.version_info[0] == 2: @timer def mapcall(N): return map((lambda x: x * 2), range(N)) else: @timer def mapcall(N): return list(map((lambda x: x * 2), range(N))) ...
__name__ == '__main__'
Adding Decorator Arguments
def timer(label=''): def decorator(func): def onCall(*args): ... print(label, ... return onCall return decorator @timer('==>') def listcomp(N): ... listcomp(...)
listcomp timer timer 1008 | Chapter 38: Decorators
decorator label
www.it-ebooks.info
import time def timer(label='', trace=True): class Timer: def __init__(self, func): self.func = func self.alltime = 0 def __call__(self, *args, **kargs): start = time.clock() result = self.func(*args, **kargs) elapsed = time.clock() - start self.alltime += elapsed if trace: format = '%s %s: %.5f, %.5f' values = (label, self.func.__name__, elapsed, self.alltime) print(format % values) return result return Timer
Timer timer Timer Timer
from mytools import timer @timer(label='[CCC]==>') def listcomp(N): return [x * 2 for x in range(N)] @timer(trace=True, label='[MMM]==>') def mapcall(N): return map((lambda x: x * 2), range(N)) for func in (listcomp, mapcall): print('') result = func(5) func(50000) func(500000) func(1000000) print(result)
Coding Function Decorators | 1009
www.it-ebooks.info
print('allTime = %s' % func.alltime) print('map/comp = %s' % round(mapcall.alltime / listcomp.alltime, 3))
map
[CCC]==> listcomp: 0.00003, [CCC]==> listcomp: 0.00640, [CCC]==> listcomp: 0.08687, [CCC]==> listcomp: 0.17911, [0, 2, 4, 6, 8] allTime = 0.272407666337 [MMM]==> mapcall: 0.00004, [MMM]==> mapcall: 0.01340, [MMM]==> mapcall: 0.13907, [MMM]==> mapcall: 0.27907, [0, 2, 4, 6, 8] allTime = 0.431572169089 map/comp = 1.584
0.00003 0.00643 0.09330 0.27241
0.00004 0.01343 0.15250 0.43157
>>> from mytools import timer >>> @timer(trace=False) ... def listcomp(N): ... return [x * 2 for x in range(N)] ... >>> x = listcomp(5000) >>> x = listcomp(5000) >>> x = listcomp(5000) >>> listcomp >>> listcomp.alltime 0.0051938863738243413 >>> @timer(trace=True, label='\t=>') ... def listcomp(N): ... return [x * 2 for x in range(N)] ... >>> x = listcomp(5000) => listcomp: 0.00155, 0.00155 >>> x = listcomp(5000) => listcomp: 0.00156, 0.00311 >>> x = listcomp(5000) => listcomp: 0.00174, 0.00486 >>> listcomp.alltime 0.0048562736325408196
1010 | Chapter 38: Decorators
list
www.it-ebooks.info
Coding Class Decorators
Singleton Classes
singleton @ instances = {} def getInstance(aClass, *args): if aClass not in instances: instances[aClass] = aClass(*args) return instances[aClass] def singleton(aClass): def onCall(*args): return getInstance(aClass, *args) return onCall
@singleton class Person: def __init__(self, name, hours, rate): self.name = name self.hours = hours
Coding Class Decorators | 1011
www.it-ebooks.info
self.rate = rate def pay(self): return self.hours * self.rate @singleton class Spam: def __init__(self, val): self.attr = val bob = Person('Bob', 40, 10) print(bob.name, bob.pay()) sue = Person('Sue', 50, 20) print(sue.name, sue.pay()) X = Spam(42) Y = Spam(99) print(X.attr, Y.attr)
Person
Spam onCall
getInstance Bob 400 Bob 400 42 42
nonlocal
def singleton(aClass): instance = None def onCall(*args): nonlocal instance if instance == None: instance = aClass(*args) return instance return onCall
class singleton: def __init__(self, aClass): self.aClass = aClass self.instance = None def __call__(self, *args):
1012 | Chapter 38: Decorators
www.it-ebooks.info
if self.instance == None: self.instance = self.aClass(*args) return self.instance
__name__ **kargs
Tracing Object Interfaces
__getattr__
__getattr__
class Wrapper: def __init__(self, object): self.wrapped = object def __getattr__(self, attrname): print('Trace:', attrname) return getattr(self.wrapped, attrname) >>> x = Wrapper([1,2,3]) >>> x.append(4) Trace: append >>> x.wrapped [1, 2, 3, 4] >>> x = Wrapper({"a": 1, "b": 2}) >>> list(x.keys()) Trace: keys ['a', 'b']
Wrapper getattr
Coding Class Decorators | 1013
www.it-ebooks.info
__getattr__
**kargs def Tracer(aClass): class Wrapper: def __init__(self, *args, **kargs): self.fetches = 0 self.wrapped = aClass(*args, **kargs) def __getattr__(self, attrname): print('Trace: ' + attrname) self.fetches += 1 return getattr(self.wrapped, attrname) return Wrapper @Tracer class Spam: def display(self): print('Spam!' * 8) @Tracer class Person: def __init__(self, name, hours, rate): self.name = name self.hours = hours self.rate = rate def pay(self): return self.hours * self.rate food = Spam() food.display() print([food.fetches]) bob = Person('Bob', 40, 50) print(bob.name) print(bob.pay()) print('') sue = Person('Sue', rate=100, hours=60) print(sue.name) print(sue.pay()) print(bob.name) print(bob.pay()) print([bob.fetches, sue.fetches])
1014 | Chapter 38: Decorators
www.it-ebooks.info
Spam Person food bob
Wrapper
__getattr__ Wrapper
Trace: display Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam! [1] Trace: name Bob Trace: pay 2000 Trace: Sue Trace: 6000 Trace: Bob Trace: 2000 [4, 2]
name pay name pay
class x
@
Wrapper
>>> from tracer import Tracer >>> @Tracer ... class MyList(list): pass >>> x = MyList([1, 2, 3]) >>> x.append(4) Trace: append >>> x.wrapped [1, 2, 3, 4] >>> WrapList = Tracer(list) >>> x = WrapList([4, 5, 6]) >>> x.append(7) Trace: append >>> x.wrapped [4, 5, 6, 7]
Coding Class Decorators | 1015
www.it-ebooks.info
@Tracer class Person: ... bob = Person('Bob', 40, 50) sue = Person('Sue', rate=100, hours=60) class Person: ... bob = Wrapper(Person('Bob', 40, 50)) sue = Wrapper(Person('Sue', rate=100, hours=60))
__getattr__ __str__ __repr__ object + __getattribute__
__getattr__
__getattr__
__repr__ Wrapper >>> x Trace: __repr__ [4, 5, 6, 7] >>> x
Private
Class Blunders II: Retaining Multiple Instances __init__ __call__ 1016 | Chapter 38: Decorators
@
www.it-ebooks.info
Tracer class Tracer: def __init__(self, aClass): self.aClass = aClass def __call__(self, *args): self.wrapped = self.aClass(*args) return self def __getattr__(self, attrname): print('Trace: ' + attrname) return getattr(self.wrapped, attrname) @Tracer class Spam: def display(self): print('Spam!' * 8) ... food = Spam() food.display()
__call__ Tracer @Tracer class Person: def __init__(self, name): self.name = name bob = Person('Bob') print(bob.name) Sue = Person('Sue') print(sue.name) print(bob.name)
Trace: name Bob Trace: name Sue Trace: name Sue
Coding Class Decorators | 1017
www.it-ebooks.info
Tracer Wrapper Tracer
Decorators Versus Manager Functions Tracer __getattr__
class Spam: ... food = Wrapper(Spam()) @Tracer class Spam: ... food = Spam()
instances = {} def getInstance(aClass, *args): if aClass not in instances: instances[aClass] = aClass(*args) return instances[aClass] bob = getInstance(Person, 'Bob', 40, 10)
instances = {} def getInstance(object): aClass = object.__class__ if aClass not in instances: instances[aClass] = object return instances[aClass] bob = getInstance(Person('Bob', 40, 10))
1018 | Chapter 38: Decorators
www.it-ebooks.info
def func(x, y): ... result = tracer(func, (1, 2)) @tracer def func(x, y): ... result = func(1, 2)
Why Decorators? (Revisited)
Coding Class Decorators | 1019
www.it-ebooks.info
@
__init__ X=Class().init() __init__
1020 | Chapter 38: Decorators
www.it-ebooks.info
Managing Functions and Classes Directly
registry = {} def register(obj): registry[obj.__name__] = obj return obj @register def spam(x): return(x ** 2) @register def ham(x): return(x ** 3) @register class Eggs: def __init__(self, x): self.data = x ** 4 def __str__(self): return str(self.data) print('Registry:') for name in registry: print(name, '=>', registry[name], type(registry[name])) print('\nManual calls:') print(spam(2)) print(ham(2)) X = Eggs(2) print(X) print('\nRegistry calls:') for name in registry: print(name, '=>', registry[name](3))
Managing Functions and Classes Directly | 1021
www.it-ebooks.info
Registry: Eggs => ham => spam => Manual calls: 4 8 16 Registry calls: Eggs => 81 ham => 27 spam => 9
def
>>> def decorate(func): ... func.marked = True ... return func ... >>> @decorate ... def spam(a, b): ... return a + b ... >>> spam.marked True >>> def annotate(text): ... def decorate(func): ... func.label = text ... return func ... return decorate ... >>> @annotate('spam data') ... def spam(a, b): ... return a + b ...
1022 | Chapter 38: Decorators
www.it-ebooks.info
>>> spam(1, 2), spam.label (3, 'spam data')
Example: “Private” and “Public” Attributes
Implementing Private Attributes Private
__getattr__
__setattr__ raise
try print
raise
""" Privacy for attributes fetched from class instances. See self-test code at end of file for a usage example. Decorator same as: Doubler = Private('data', 'size')(Doubler). Private returns onDecorator, onDecorator returns onInstance, and each onInstance instance embeds a Doubler instance. """
Example: “Private” and “Public” Attributes | 1023
www.it-ebooks.info
traceMe = False def trace(*args): if traceMe: print('[' + ' '.join(map(str, args)) + ']') def Private(*privates): def onDecorator(aClass): class onInstance: def __init__(self, *args, **kargs): self.wrapped = aClass(*args, **kargs) def __getattr__(self, attr): trace('get:', attr) if attr in privates: raise TypeError('private attribute fetch: ' + attr) else: return getattr(self.wrapped, attr) def __setattr__(self, attr, value): trace('set:', attr, value) if attr == 'wrapped': self.__dict__[attr] = value elif attr in privates: raise TypeError('private attribute change: ' + attr) else: setattr(self.wrapped, attr, value) return onInstance return onDecorator if __name__ == '__main__': traceMe = True @Private('data', 'size') class Doubler: def __init__(self, label, start): self.label = label self.data = start def size(self): return len(self.data) def double(self): for i in range(self.size()): self.data[i] = self.data[i] * 2 def display(self): print('%s => %s' % (self.label, self.data)) X = Doubler('X is', [1, 2, 3]) Y = Doubler('Y is', [−10, −20, −30]) # The followng all succeed print(X.label) X.display(); X.double(); X.display() print(Y.label) Y.display(); Y.double() Y.label = 'Spam' Y.display()
1024 | Chapter 38: Decorators
www.it-ebooks.info
""" print(X.size()) print(X.data) X.data = [1, 1, 1] X.size = lambda S: 0 print(Y.data) print(Y.size()) """
traceMe
True
[set: wrapped ] [set: wrapped ] [get: label] X is [get: display] X is => [1, 2, 3] [get: double] [get: display] X is => [2, 4, 6] [get: label] Y is [get: display] Y is => [−10, −20, −30] [get: double] [set: label Spam] [get: display] Spam => [−20, −40, −60]
Implementation Details I
Inheritance versus delegation __setattr__
__dict__
Example: “Private” and “Public” Attributes | 1025
www.it-ebooks.info
self
Decorator arguments. Private
Private Private
State retention and enclosing scopes
Private onDecorator
onInstance
onDecorator onInstance
Using __dict__ and __slots__ __setattr__
__dict__ onInstance
setattr getattr
__dict__
__slots__ __dict__ setattr __dict__
wrapped
__dict__ onInstance getattr
__slots__
Generalizing for Public Declarations, Too Private Public
1026 | Chapter 38: Decorators
Private
www.it-ebooks.info
Private
Public
Private Private Private Public Public Private Private
Public Public Public
Public
Private Private Public
lambda
""" Class decorator with Private and Public attribute declarations. Controls access to attributes stored on an instance, or inherited by it from its classes. Private declares attribute names that cannot be fetched or assigned outside the decorated class, and Public declares all the names that can. Caveat: this works in 3.0 for normally named attributes only: __X__ operator overloading methods implicitly run for built-in operations do not trigger either __getattr__ or __getattribute__ in new-style classes. Add __X__ methods here to intercept and delegate built-ins. """ traceMe = False def trace(*args): if traceMe: print('[' + ' '.join(map(str, args)) + ']') def accessControl(failIf): def onDecorator(aClass): class onInstance: def __init__(self, *args, **kargs): self.__wrapped = aClass(*args, **kargs) def __getattr__(self, attr): trace('get:', attr)
Example: “Private” and “Public” Attributes | 1027
www.it-ebooks.info
if failIf(attr): raise TypeError('private attribute fetch: ' + attr) else: return getattr(self.__wrapped, attr) def __setattr__(self, attr, value): trace('set:', attr, value) if attr == '_onInstance__wrapped': self.__dict__[attr] = value elif failIf(attr): raise TypeError('private attribute change: ' + attr) else: setattr(self.__wrapped, attr, value) return onInstance return onDecorator def Private(*attributes): return accessControl(failIf=(lambda attr: attr in attributes)) def Public(*attributes): return accessControl(failIf=(lambda attr: attr not in attributes))
Private Public Private Public >>> from access import Private, Public >>> @Private('age') ... class Person: ... def __init__(self, name, age): ... self.name = name ... self.age = age ... >>> X = Person('Bob', 40) >>> X.name 'Bob' >>> X.name = 'Sue' >>> X.name 'Sue' >>> X.age TypeError: private attribute fetch: age >>> X.age = 'Tom' TypeError: private attribute change: age >>> @Public('name') ... class Person: ... def __init__(self, name, age): ... self.name = name ... self.age = age ... >>> X = Person('bob', 40) >>> X.name 'bob' >>> X.name = 'Sue'
1028 | Chapter 38: Decorators
www.it-ebooks.info
>>> X.name 'Sue' >>> X.age TypeError: private attribute fetch: age >>> X.age = 'Tom' TypeError: private attribute change: age
Implementation Details II
Using __X pseudoprivate names __X wrapped wrapped
'_onInstance__wrapped'
__setattr__
Breaking privacy
wrapped bob._onInstance__wrapped.pay
bob.pay
#define private public
Decorator tradeoffs
X.__class__
isinstance(X, C)
Example: “Private” and “Public” Attributes | 1029
www.it-ebooks.info
Open Issues
Caveat: operator overloading methods fail to delegate under 3.0 __getattr__ __str__
__add__
__X__ __getattr__ __getattribute__ onInstance.__getattr__ object
onInstance
__str__ __add__ __getattr__ C:\misc> c:\python26\python >>> from access import Private >>> @Private('age') ... class Person: ... def __init__(self): ... self.age = 42 ... def __str__(self): ... return 'Person: ' + str(self.age) ... def __add__(self, yrs): ... self.age += yrs ...
1030 | Chapter 38: Decorators
+ Person
www.it-ebooks.info
>>> X = Person() >>> X.age TypeError: private attribute fetch: age >>> print(X) Person: 42 >>> X + 10 >>> print(X) Person: 52
__str__ __add__
__getattr__ print object
+
C:\misc> c:\python30\python >>> from access import Private >>> @Private('age') ... class Person: ... def __init__(self): ... self.age = 42 ... def __str__(self): ... return 'Person: ' + str(self.age) ... def __add__(self, yrs): ... self.age += yrs ... >>> X = Person() >>> X.age TypeError: private attribute fetch: age >>> print(X) >>> X + 10 TypeError: unsupported operand type(s) for +: 'onInstance' and 'int' >>> print(X)
__getattribute__ property
Private
Public
failIf
Example: “Private” and “Public” Attributes | 1031
www.it-ebooks.info
def accessControl(failIf): def onDecorator(aClass): class onInstance: def __init__(self, *args, **kargs): self.__wrapped = aClass(*args, **kargs) def __str__(self): return str(self.__wrapped) def __add__(self, other): return self.__wrapped + other def __getitem__(self, index): return self.__wrapped[index] def __call__(self, *args, **kargs): return self.__wrapped(*arg, *kargs) ...plus any others needed... def __getattr__(self, attr): ... def __setattr__(self, attr, value): ... return onInstance return onDecorator
__str__ __add__
Implementation alternatives: __getattribute__ inserts, call stack inspection
1032 | Chapter 38: Decorators
www.it-ebooks.info
__getattribute__
def accessControl(failIf): def onDecorator(aClass): def getattributes(self, attr): trace('get:', attr) if failIf(attr): raise TypeError('private attribute fetch: ' + attr) else: return object.__getattribute__(self, attr) aClass.__getattribute__ = getattributes return aClass return onDecorator def Private(*attributes): return accessControl(failIf=(lambda attr: attr in attributes)) def Public(*attributes): return accessControl(failIf=(lambda attr: attr not in attributes))
__setattr__
__setattr__ __setattr__
Person
__getattribute__ __slots__
__getattribute__ __str__ __str__ __getattribute__ __str__
__X__
Private
__getattribute__
Example: “Private” and “Public” Attributes | 1033
www.it-ebooks.info
Private
Python Isn’t About Control Private
Example: Validating Function Arguments
The Goal class Person: ...
1034 | Chapter 38: Decorators
Public
www.it-ebooks.info
def giveRaise(self, percent): self.pay = int(self.pay * (1 + percent))
if
assert
class Person: def giveRaise(self, percent): if percent < 0.0 or percent > 1.0: raise TypeError, 'percent invalid' self.pay = int(self.pay * (1 + percent)) class Person: def giveRaise(self, percent): assert percent >= 0.0 and percent high: errmsg = 'Argument %s not in %s..%s' % (ix, low, high) raise TypeError(errmsg) return func(*args) return onCall return onDecorator
def onCall *args
True main.py
__debug__
__debug__ –O False
from devtools import rangetest print(__debug__) @rangetest((1, 0, 120)) def persinfo(name, age): print('%s is %s years old' % (name, age)) @rangetest([0, 1, 12], [1, 1, 31], [2, 0, 2009]) def birthday(M, D, Y): print('birthday = {0}/{1}/{2}'.format(M, D, Y)) class Person: def __init__(self, name, job, pay): self.job = job self.pay = pay @rangetest([1, 0.0, 1.0]) def giveRaise(self, percent): self.pay = int(self.pay * (1 + percent))
persinfo('Bob Smith', 45) birthday(5, 31, 1963)
1036 | Chapter 38: Decorators
self
python –O
www.it-ebooks.info
sue = Person('Sue Jones', 'dev', 100000) sue.giveRaise(.10) print(sue.pay)
print C:\misc> C:\python30\python devtools_test.py True Bob Smith is 45 years old birthday = 5/31/1963 110000
TypeError
C:\misc> C:\python30\python devtools_test.py True Bob Smith is 45 years old birthday = 5/31/1963 110000 TypeError: Argument 1 not in 0.0..1.0
–O
C:\misc> C:\python30\python –O devtools_test.py False Bob Smith is 45 years old birthday = 5/31/1963 110000 231000
Generalizing for Keywords and Defaults, Too
Example: Validating Function Arguments | 1037
www.it-ebooks.info
*pargs
**kargs
""" File devtools.py: function decorator that performs range-test validation for passed arguments. Arguments are specified by keyword to the decorator. In the actual call, arguments may be passed by position or keyword, and defaults may be omitted. See devtools_test.py for example use cases. """ trace = True def rangetest(**argchecks): def onDecorator(func): if not __debug__: return func else: import sys code = func.__code__ allargs = code.co_varnames[:code.co_argcount] funcname = func.__name__ def onCall(*pargs, **kargs): positionals = list(allargs) positionals = positionals[:len(pargs)] for (argname, (low, high)) in argchecks.items(): if argname in kargs: if kargs[argname] < low or kargs[argname] > high: errmsg = '{0} argument "{1}" not in {2}..{3}' errmsg = errmsg.format(funcname, argname, low, high) raise TypeError(errmsg) elif argname in positionals: position = positionals.index(argname) if pargs[position] < low or pargs[position] > high: errmsg = '{0} argument "{1}" not in {2}..{3}' errmsg = errmsg.format(funcname, argname, low, high) raise TypeError(errmsg) else: if trace: print('Argument "{0}" defaulted'.format(argname))
1038 | Chapter 38: Decorators
www.it-ebooks.info
return func(*pargs, **kargs) return onCall return onDecorator
from devtools import rangetest
@rangetest(age=(0, 120)) def persinfo(name, age): print('%s is %s years old' % (name, age)) @rangetest(M=(1, 12), D=(1, 31), Y=(0, 2009)) def birthday(M, D, Y): print('birthday = {0}/{1}/{2}'.format(M, D, Y)) persinfo('Bob', 40) persinfo(age=40, name='Bob') birthday(5, D=1, Y=1963)
class Person: def __init__(self, name, job, pay): self.job = job self.pay = pay @rangetest(percent=(0.0, 1.0)) def giveRaise(self, percent): self.pay = int(self.pay * (1 + percent)) bob = Person('Bob Smith', 'dev', 100000) sue = Person('Sue Jones', 'dev', 100000) bob.giveRaise(.10) sue.giveRaise(percent=.20) print(bob.pay, sue.pay)
@rangetest(a=(1, 10), b=(1, 10), c=(1, 10), d=(1, 10)) def omitargs(a, b=7, c=8, d=9):
Example: Validating Function Arguments | 1039
www.it-ebooks.info
print(a, b, c, d) omitargs(1, 2, 3, 4) omitargs(1, 2, 3) omitargs(1, 2, 3, d=4) omitargs(1, d=4) omitargs(d=4, a=1) omitargs(1, b=2, d=4) omitargs(d=8, c=7, a=1)
C:\misc> C:\python30\python devtools_test.py Bob is 40 years old Bob is 40 years old birthday = 5/1/1963 110000 120000 1 2 3 4 Argument "d" defaulted 1 2 3 9 1 2 3 4 Argument "c" defaulted Argument "b" defaulted 1 7 8 4 Argument "c" defaulted Argument "b" defaulted 1 7 8 4 Argument "c" defaulted 1 2 8 4 Argument "b" defaulted 1 7 7 8
–O TypeError: giveRaise argument "percent" not in 0.0..1.0
Implementation Details
1040 | Chapter 38: Decorators
www.it-ebooks.info
Function introspection
>>> def func(a, b, c, d): ... x = 1 ... y = 2 ... >>> code = func.__code__ >>> code.co_nlocals 6 >>> code.co_varnames ('a', 'b', 'c', 'd', 'x', 'y') >>> code.co_varnames[:code.co_argcount] ('a', 'b', 'c', 'd') >>> >>> (3, >>>
import sys sys.version_info 0, 0, 'final', 0) code = func.__code__ if sys.version_info[0] == 3 else func.func_code
func.__code__ __code__
func.func_code dir
Argument assumptions
def
Example: Validating Function Arguments | 1041
www.it-ebooks.info
Matching algorithm
*pargs
*pargs
**kargs *pargs
*pargs **kargs
Open Issues
omitargs() omitargs(d=8, c=7, b=6)
1042 | Chapter 38: Decorators
www.it-ebooks.info
*args
**args
**kargs **kargs
**kargs
*args *pargs
Decorator Arguments Versus Function Annotations
def
@rangetest(a=(1, 5), c=(0.0, 1.0)) def func(a, b, c): print(a + b + c)
Example: Validating Function Arguments | 1043
www.it-ebooks.info
@rangetest def func(a:(1, 5), b, c:(0.0, 1.0)): print(a + b + c)
def rangetest(**argchecks): def onDecorator(func): def onCall(*pargs, **kargs): print(argchecks) for check in argchecks: pass return func(*pargs, **kargs) return onCall return onDecorator @rangetest(a=(1, 5), c=(0.0, 1.0)) def func(a, b, c): print(a + b + c) func(1, 2, c=3)
def rangetest(func): def onCall(*pargs, **kargs): argchecks = func.__annotations__ print(argchecks) for check in argchecks: pass return func(*pargs, **kargs) return onCall @rangetest def func(a:(1, 5), b, c:(0.0, 1.0)): print(a + b + c) func(1, 2, c=3)
{'a': (1, 5), 'c': (0.0, 1.0)} 6 {'a': (1, 5), 'c': (0.0, 1.0)} 6
1044 | Chapter 38: Decorators
www.it-ebooks.info
def
Other Applications: Type Testing (If You Insist!)
def typetest(**argchecks): def onDecorator(func): .... def onCall(*pargs, **kargs): positionals = list(allargs)[:len(pargs)] for (argname, type) in argchecks.items(): if argname in kargs: if not isinstance(kargs[argname], type): ... raise TypeError(errmsg) elif argname in positionals: position = positionals.index(argname) if not isinstance(pargs[position], type): ... raise TypeError(errmsg)
Example: Validating Function Arguments | 1045
www.it-ebooks.info
else: return func(*pargs, **kargs) return onCall return onDecorator @typetest(a=int, c=float) def func(a, b, c, d): ... func(1, 2, 3.0, 4) func('spam', 2, 99, 4)
Public
@typetest def func(a: int, b, c: float, d): ...
Chapter Summary
1046 | Chapter 38: Decorators
www.it-ebooks.info
Test Your Knowledge: Quiz __call__ self
Public Private @ __debug__
–O
python –O main.py...
Test Your Knowledge: Answers self
import time def timer(label='', trace=True): def onDecorator(func): def onCall(*args, **kargs): start = time.clock() result = func(*args, **kargs) elapsed = time.clock() - start
Test Your Knowledge: Answers | 1047
www.it-ebooks.info
onCall.alltime += elapsed if trace: format = '%s%s: %.5f, %.5f' values = (label, func.__name__, elapsed, onCall.alltime) print(format % values) return result onCall.alltime = 0 return onCall return onDecorator
@timer(trace=True, label='[CCC]==>') def listcomp(N): return [x * 2 for x in range(N)] @timer(trace=True, label='[MMM]==>') def mapcall(N): return list(map((lambda x: x * 2), range(N))) for func in (listcomp, mapcall): result = func(5) func(5000000) print(result) print('allTime = %s\n' % func.alltime)
class Person: def __init__(self, name, pay): self.name = name self.pay = pay @timer() def giveRaise(self, percent): self.pay *= (1.0 + percent) @timer(label='**') def lastName(self): return self.name.split()[-1] bob = Person('Bob Smith', 50000) sue = Person('Sue Jones', 100000) bob.giveRaise(.10) sue.giveRaise(.20) print(bob.pay, sue.pay) print(bob.lastName(), sue.lastName()) print('%.5f %.5f' % (Person.giveRaise.alltime, Person.lastName.alltime))
[CCC]==>listcomp: 0.00002, 0.00002 [CCC]==>listcomp: 1.19636, 1.19638 [0, 2, 4, 6, 8] allTime = 1.19637775192
1048 | Chapter 38: Decorators
www.it-ebooks.info
[MMM]==>mapcall: 0.00002, 0.00002 [MMM]==>mapcall: 2.29260, 2.29262 [0, 2, 4, 6, 8] allTime = 2.2926232943 giveRaise: 0.00001, 0.00001 giveRaise: 0.00001, 0.00002 55000.0 120000.0 **lastName: 0.00001, 0.00001 **lastName: 0.00001, 0.00002 Smith Jones 0.00002 0.00002
–O
__getattr__ traceMe = False def trace(*args): if traceMe: print('[' + ' '.join(map(str, args)) + ']') def accessControl(failIf): def onDecorator(aClass): if not __debug__: return aClass else: class onInstance: def __init__(self, *args, **kargs): self.__wrapped = aClass(*args, **kargs) def __getattr__(self, attr): trace('get:', attr) if failIf(attr): raise TypeError('private attribute fetch: ' + attr) else: return getattr(self.__wrapped, attr) def __setattr__(self, attr, value): trace('set:', attr, value) if attr == '_onInstance__wrapped': self.__dict__[attr] = value elif failIf(attr): raise TypeError('private attribute change: ' + attr) else: setattr(self.__wrapped, attr, value) return onInstance return onDecorator def Private(*attributes): return accessControl(failIf=(lambda attr: attr in attributes)) def Public(*attributes): return accessControl(failIf=(lambda attr: attr not in attributes))
Test Your Knowledge: Answers | 1049
www.it-ebooks.info
@Private('age') class Person: def __init__(self, name, age): self.name = name self.age = age X = Person('Bob', 40) print(X.name) X.name = 'Sue' print(X.name) #print(X.age) #X.age = 999 #print(X.age) @Public('name') class Person: def __init__(self, name, age): self.name = name self.age = age X = Person('bob', 40) print(X.name) X.name = 'Sue' print(X.name) #print(X.age) #X.age = 999 #print(X.age)
1050 | Chapter 38: Decorators
www.it-ebooks.info
CHAPTER 39
Metaclasses
class
1051
www.it-ebooks.info
To Metaclass or Not to Metaclass
Increasing Levels of Magic
1052 | Chapter 39: Metaclasses
www.it-ebooks.info
__class__
__dict__
__str__
__add__
__getattr__
__setattr__
__getattribute__
property
property __get__ __set__
__delete__
@callable
To Metaclass or Not to Metaclass | 1053
www.it-ebooks.info
class
The Downside of “Helper” Functions
1054 | Chapter 39: Metaclasses
www.it-ebooks.info
class Extras: def extra(self, args): ... class Client1(Extras): ... class Client2(Extras): ... class Client3(Extras): ... X = Client1() X.extra()
required def extra(self, arg): ... class Client1: ... if required(): Client1.extra = extra class Client2: ... if required(): Client2.extra = extra class Client3: ... if required(): Client3.extra = extra X = Client1() X.extra()
class self
def extra(self, arg): ... def extras(Class): if required(): Class.extra = extra class Client1: ...
To Metaclass or Not to Metaclass | 1055
www.it-ebooks.info
extras(Client1) class Client2: ... extras(Client2) class Client3: ... extras(Client3) X = Client1() X.extra()
class
def extra(self, arg): ... class Extras(type): def __init__(Class, classname, superclasses, attributedict): if required(): Class.extra = extra class Client1(metaclass=Extras): ... class Client2(metaclass=Extras): ... class Client3(metaclass=Extras): ... X = Client1() X.extra()
class
Metaclasses Versus Class Decorators: Round 1
1056 | Chapter 39: Metaclasses
www.it-ebooks.info
def extra(self, arg): ... def extras(Class): if required(): Class.extra = extra return Class class Client1: ... Client1 = extras(Client1) class Client2: ... Client2 = extras(Client2) class Client3: ... Client3 = extras(Client3) X = Client1() X.extra()
def extra(self, arg): ... def extras(Class): if required(): Class.extra = extra return Class @extras class Client1: ... @extras class Client2: ... @extras class Client3: ... X = Client1() X.extra()
To Metaclass or Not to Metaclass | 1057
www.it-ebooks.info
__init__
The Metaclass Model class
Classes Are Instances of type
type object type
1058 | Chapter 39: Metaclasses
type
www.it-ebooks.info
type type
type
C:\misc> c:\python30\python >>> type([]) >>> type(type([])) >>> type(list) >>> type(type)
type C:\misc> c:\python26\python >>> type([]) >>> type(type([])) >>> type(list) >>> type(type)
type
type
The Metaclass Model | 1059
www.it-ebooks.info
type __class__
type
__class__
C:\misc> c:\python30\python >>> class C: pass ... >>> X = C() >>> type(X) >>> X.__class__ >>> type(C) >>> C.__class__
type
type object C:\misc> c:\python26\python >>> class C(object): pass ... >>> X = C() >>> type(X) >>> type(C) >>> X.__class__ >>> C.__class__
__class__ type C:\misc> c:\python26\python >>> class C: pass ... >>> X = C()
1060 | Chapter 39: Metaclasses
www.it-ebooks.info
>>> type(X) >>> type(C) >>> X.__class__ >>> C.__class__ AttributeError: class C has no attribute '__class__'
Metaclasses Are Subclasses of Type type type type type
type type type
type
class __bases__
Class Statement Protocol type type class
The Metaclass Model | 1061
www.it-ebooks.info
def
class type
class
class = type(classname, superclasses, attributedict)
type
__call__ type
type.__new__(typeclass, classname, superclasses, attributedict) type.__init__(class, classname, superclasses, attributedict)
__new__
class
__init__
type class Spam(Eggs): data = 1 def meth(self, arg): pass
data meth class
type
class
Spam = type('Spam', (Eggs,), {'data': 1, 'meth': meth, '__module__': '__main__'})
class type
Declaring Metaclasses type
class class Spam(metaclass=Meta):
Spam Meta class Spam(Eggs, metaclass=Meta):
1062 | Chapter 39: Metaclasses
Eggs
www.it-ebooks.info
object
class spam(object): __metaclass__ = Meta
__metaclass__
object class class
type
class = Meta(classname, superclasses, attributedict)
type
type
__call__
class Meta.__new__(Meta, classname, superclasses, attributedict) Meta.__init__(class, classname, superclasses, attributedict)
class Spam(Eggs, metaclass=Meta): data = 1 def meth(self, arg): pass
class class Spam = Meta('Spam', (Eggs,), {'data': 1, 'meth': meth, '__module__': '__main__'})
__new__ __init__ type __call__
Coding Metaclasses type class class type Coding Metaclasses | 1063
www.it-ebooks.info
A Basic Metaclass type __new__
type __new__
__call__
type type
__new__ class Meta(type): def __new__(meta, classname, supers, classdict): return type.__new__(meta, classname, supers, classdict)
type
type
__call__
__new__
class __init__
class MetaOne(type): def __new__(meta, classname, supers, classdict): print('In MetaOne.new:', classname, supers, classdict, sep='\n...') return type.__new__(meta, classname, supers, classdict) class Eggs: pass print('making class') class Spam(Eggs, metaclass=MetaOne): data = 1 def meth(self, arg): pass print('making instance') X = Spam() print('data:', X.data)
Spam
Eggs
MetaOne
X
Spam class making class In MetaOne.new: ...Spam ...(,) ...{'__module__': '__main__', 'data': 1, 'meth': } making instance data: 1
1064 | Chapter 39: Metaclasses
www.it-ebooks.info
Customizing Construction and Initialization __init__ __call__
__new__
__init__
class MetaOne(type): def __new__(meta, classname, supers, classdict): print('In MetaOne.new: ', classname, supers, classdict, sep='\n...') return type.__new__(meta, classname, supers, classdict) def __init__(Class, classname, supers, classdict): print('In MetaOne init:', classname, supers, classdict, sep='\n...') print('...init class object:', list(Class.__dict__.keys())) class Eggs: pass print('making class') class Spam(Eggs, metaclass=MetaOne): data = 1 def meth(self, arg): pass print('making instance') X = Spam() print('data:', X.data)
class making class In MetaOne.new: ...Spam ...(,) ...{'__module__': '__main__', 'data': 1, 'meth': } In MetaOne init: ...Spam ...(,) ...{'__module__': '__main__', 'data': 1, 'meth': } ...init class object: ['__module__', 'data', 'meth', '__doc__'] making instance data: 1
Other Metaclass Coding Techniques type
__new__
__init__
Coding Metaclasses | 1065
www.it-ebooks.info
Using simple factory functions class
def MetaFunc(classname, supers, classdict): print('In MetaFunc: ', classname, supers, classdict, sep='\n...') return type(classname, supers, classdict) class Eggs: pass print('making class') class Spam(Eggs, metaclass=MetaFunc): data = 1 def meth(self, args): pass print('making instance') X = Spam() print('data:', X.data)
class type
__call__
making class In MetaFunc: ...Spam ...(,) ...{'__module__': '__main__', 'data': 1, 'meth': } making instance data: 1
Overloading class creation calls with metaclasses class __call__
class SuperMeta(type): def __call__(meta, classname, supers, classdict): print('In SuperMeta.call: ', classname, supers, classdict, sep='\n...') return type.__call__(meta, classname, supers, classdict) class SubMeta(type, metaclass=SuperMeta): def __new__(meta, classname, supers, classdict):
1066 | Chapter 39: Metaclasses
type
www.it-ebooks.info
print('In SubMeta.new: ', classname, supers, classdict, sep='\n...') return type.__new__(meta, classname, supers, classdict) def __init__(Class, classname, supers, classdict): print('In SubMeta init:', classname, supers, classdict, sep='\n...') print('...init class object:', list(Class.__dict__.keys())) class Eggs: pass print('making class') class Spam(Eggs, metaclass=SubMeta): data = 1 def meth(self, arg): pass print('making instance') X = Spam() print('data:', X.data)
type making class In SuperMeta.call: ...Spam ...(,) ...{'__module__': '__main__', 'data': 1, 'meth': } In SubMeta.new: ...Spam ...(,) ...{'__module__': '__main__', 'data': 1, 'meth': } In SubMeta init: ...Spam ...(,) ...{'__module__': '__main__', 'data': 1, 'meth': } ...init class object: ['__module__', 'data', 'meth', '__doc__'] making instance data: 1
Overloading class creation calls with normal classes
__call__
__new__ SubMeta
__init__
Coding Metaclasses | 1067
www.it-ebooks.info
class SuperMeta: def __call__(self, classname, supers, classdict): print('In SuperMeta.call: ', classname, supers, classdict, sep='\n...') Class = self.__New__(classname, supers, classdict) self.__Init__(Class, classname, supers, classdict) return Class class SubMeta(SuperMeta): def __New__(self, classname, supers, classdict): print('In SubMeta.new: ', classname, supers, classdict, sep='\n...') return type(classname, supers, classdict) def __Init__(self, Class, classname, supers, classdict): print('In SubMeta init:', classname, supers, classdict, sep='\n...') print('...init class object:', list(Class.__dict__.keys())) class Eggs: pass print('making class') class Spam(Eggs, metaclass=SubMeta()): data = 1 def meth(self, arg): pass print('making instance') X = Spam() print('data:', X.data)
type
__new__
__init__
Instances Versus Inheritance
class type type
__new__
__init__ __call__ type
1068 | Chapter 39: Metaclasses
www.it-ebooks.info
metaclass=M
__dict__
class MetaOne(type): def __new__(meta, classname, supers, classdict): print('In MetaOne.new:', classname) return type.__new__(meta, classname, supers, classdict) def toast(self): print('toast') class Super(metaclass=MetaOne): def spam(self): print('spam') class C(Super): def eggs(self): print('eggs') X = C() X.eggs() X.spam() X.toast()
In MetaOne.new: Super In MetaOne.new: C eggs spam AttributeError: 'C' object has no attribute 'toast'
Coding Metaclasses | 1069
www.it-ebooks.info
Example: Adding Methods to Classes
Manual Augmentation
class Client1: def __init__(self, value): self.value = value def spam(self): return self.value * 2 class Client2: value = 'ni?' def eggsfunc(obj): return obj.value * 4 def hamfunc(obj, value): return value + 'ham' Client1.eggs = eggsfunc Client1.ham = hamfunc Client2.eggs = eggsfunc
1070 | Chapter 39: Metaclasses
www.it-ebooks.info
Client2.ham
= hamfunc
X = Client1('Ni!') print(X.spam()) print(X.eggs()) print(X.ham('bacon')) Y = Client2() print(Y.eggs()) print(Y.ham('bacon'))
self
Ni!Ni! Ni!Ni!Ni!Ni! baconham ni?ni?ni?ni? baconham
Metaclass-Based Augmentation
def eggsfunc(obj): return obj.value * 4
Example: Adding Methods to Classes | 1071
www.it-ebooks.info
def hamfunc(obj, value): return value + 'ham' class Extender(type): def __new__(meta, classname, supers, classdict): classdict['eggs'] = eggsfunc classdict['ham'] = hamfunc return type.__new__(meta, classname, supers, classdict) class Client1(metaclass=Extender): def __init__(self, value): self.value = value def spam(self): return self.value * 2 class Client2(metaclass=Extender): value = 'ni?' X = Client1('Ni!') print(X.spam()) print(X.eggs()) print(X.ham('bacon')) Y = Client2() print(Y.eggs()) print(Y.ham('bacon'))
Ni!Ni! Ni!Ni!Ni!Ni! baconham ni?ni?ni?ni? baconham
class MetaExtend(type): def __new__(meta, classname, supers, classdict): if sometest(): classdict['eggs'] = eggsfunc1 else: classdict['eggs'] = eggsfunc2 if someothertest():
1072 | Chapter 39: Metaclasses
www.it-ebooks.info
classdict['ham'] = hamfunc else: classdict['ham'] = lambda *args: 'Not supported' return type.__new__(meta, classname, supers, classdict)
Metaclasses Versus Class Decorators: Round 2
class class
Decorator-based augmentation
__init__
def eggsfunc(obj): return obj.value * 4 def hamfunc(obj, value): return value + 'ham' def Extender(aClass): aClass.eggs = eggsfunc aClass.ham = hamfunc return aClass @Extender class Client1: def __init__(self, value): self.value = value def spam(self): return self.value * 2
Example: Adding Methods to Classes | 1073
www.it-ebooks.info
@Extender class Client2: value = 'ni?' X = Client1('Ni!') print(X.spam()) print(X.eggs()) print(X.ham('bacon')) Y = Client2() print(Y.eggs()) print(Y.ham('bacon'))
Managing instances instead of classes
def Tracer(aClass): class Wrapper: def __init__(self, *args, **kargs): self.wrapped = aClass(*args, **kargs) def __getattr__(self, attrname): print('Trace:', attrname) return getattr(self.wrapped, attrname) return Wrapper @Tracer class Person: def __init__(self, name, hours, rate): self.name = name self.hours = hours self.rate = rate def pay(self): return self.hours * self.rate bob = Person('Bob', 40, 50)
1074 | Chapter 39: Metaclasses
www.it-ebooks.info
print(bob.name) print(bob.pay())
Trace: name Bob Trace: pay 2000
def Tracer(classname, supers, classdict): aClass = type(classname, supers, classdict) class Wrapper: def __init__(self, *args, **kargs): self.wrapped = aClass(*args, **kargs) def __getattr__(self, attrname): print('Trace:', attrname) return getattr(self.wrapped, attrname) return Wrapper class Person(metaclass=Tracer): def __init__(self, name, hours, rate): self.name = name self.hours = hours self.rate = rate def pay(self): return self.hours * self.rate bob = Person('Bob', 40, 50) print(bob.name) print(bob.pay())
type type
class
Trace: name Bob Trace: pay 2000
Example: Adding Methods to Classes | 1075
www.it-ebooks.info
Example: Applying Decorators to Methods class
Tracing with Decoration Manually
def tracer(func): calls = 0 def onCall(*args, **kwargs): nonlocal calls calls += 1 print('call %s to %s' % (calls, func.__name__)) return func(*args, **kwargs) return onCall import time def timer(label='', trace=True): def onDecorator(func): def onCall(*args, **kargs): start = time.clock() result = func(*args, **kargs) elapsed = time.clock() - start onCall.alltime += elapsed if trace: format = '%s%s: %.5f, %.5f' values = (label, func.__name__, elapsed, onCall.alltime) print(format % values)
1076 | Chapter 39: Metaclasses
www.it-ebooks.info
return result onCall.alltime = 0 return onCall return onDecorator
@ from mytools import tracer class Person: @tracer def __init__(self, name, pay): self.name = name self.pay = pay @tracer def giveRaise(self, percent): self.pay *= (1.0 + percent) @tracer def lastName(self): return self.name.split()[-1] bob = Person('Bob Smith', 50000) sue = Person('Sue Jones', 100000) print(bob.name, sue.name) sue.giveRaise(.10) print(sue.pay) print(bob.lastName(), sue.lastName())
call 1 to __init__ call 2 to __init__ Bob Smith Sue Jones call 1 to giveRaise 110000.0 call 1 to lastName call 2 to lastName Smith Jones
Tracing with Metaclasses and Decorators
Example: Applying Decorators to Methods | 1077
www.it-ebooks.info
from types import FunctionType from mytools import tracer class MetaTrace(type): def __new__(meta, classname, supers, classdict): for attr, attrval in classdict.items(): if type(attrval) is FunctionType: classdict[attr] = tracer(attrval) return type.__new__(meta, classname, supers, classdict) class Person(metaclass=MetaTrace): def __init__(self, name, pay): self.name = name self.pay = pay def giveRaise(self, percent): self.pay *= (1.0 + percent) def lastName(self): return self.name.split()[-1] bob = Person('Bob Smith', 50000) sue = Person('Sue Jones', 100000) print(bob.name, sue.name) sue.giveRaise(.10) print(sue.pay) print(bob.lastName(), sue.lastName())
call 1 to __init__ call 2 to __init__ Bob Smith Sue Jones call 1 to giveRaise 110000.0 call 1 to lastName call 2 to lastName Smith Jones
1078 | Chapter 39: Metaclasses
www.it-ebooks.info
Applying Any Decorator to Methods
from types import FunctionType from mytools import tracer, timer def decorateAll(decorator): class MetaDecorate(type): def __new__(meta, classname, supers, classdict): for attr, attrval in classdict.items(): if type(attrval) is FunctionType: classdict[attr] = decorator(attrval) return type.__new__(meta, classname, supers, classdict) return MetaDecorate class Person(metaclass=decorateAll(tracer)): def __init__(self, name, pay): self.name = name self.pay = pay def giveRaise(self, percent): self.pay *= (1.0 + percent) def lastName(self): return self.name.split()[-1] bob = Person('Bob Smith', 50000) sue = Person('Sue Jones', 100000) print(bob.name, sue.name) sue.giveRaise(.10) print(sue.pay) print(bob.lastName(), sue.lastName())
call 1 to __init__ call 2 to __init__ Bob Smith Sue Jones call 1 to giveRaise 110000.0 call 1 to lastName call 2 to lastName Smith Jones
class
Example: Applying Decorators to Methods | 1079
www.it-ebooks.info
class Person(metaclass=decorateAll(tracer)): class Person(metaclass=decorateAll(timer())): class Person(metaclass=decorateAll(timer(label='**'))):
print('-'*40) print('%.5f' % Person.__init__.alltime) print('%.5f' % Person.giveRaise.alltime) print('%.5f' % Person.lastName.alltime)
**__init__: 0.00001, 0.00001 **__init__: 0.00001, 0.00002 Bob Smith Sue Jones **giveRaise: 0.00001, 0.00001 110000.0 **lastName: 0.00001, 0.00001 **lastName: 0.00001, 0.00002 Smith Jones ---------------------------------------0.00002 0.00001 0.00002
Metaclasses Versus Class Decorators: Round 3
from types import FunctionType from mytools import tracer, timer def decorateAll(decorator): def DecoDecorate(aClass):
1080 | Chapter 39: Metaclasses
www.it-ebooks.info
for attr, attrval in aClass.__dict__.items(): if type(attrval) is FunctionType: setattr(aClass, attr, decorator(attrval)) return aClass return DecoDecorate @decorateAll(tracer) class Person: def __init__(self, name, pay): self.name = name self.pay = pay def giveRaise(self, percent): self.pay *= (1.0 + percent) def lastName(self): return self.name.split()[-1] bob = Person('Bob Smith', 50000) sue = Person('Sue Jones', 100000) print(bob.name, sue.name) sue.giveRaise(.10) print(sue.pay) print(bob.lastName(), sue.lastName())
call 1 to __init__ call 2 to __init__ Bob Smith Sue Jones call 1 to giveRaise 110000.0 call 1 to lastName call 2 to lastName Smith Jones
Person Person
Example: Applying Decorators to Methods | 1081
www.it-ebooks.info
@decorateAll(tracer) @decorateAll(timer()) @decorateAll(timer(label='@@'))
print('-'*40) print('%.5f' % Person.__init__.alltime) print('%.5f' % Person.giveRaise.alltime) print('%.5f' % Person.lastName.alltime)
@@__init__: 0.00001, 0.00001 @@__init__: 0.00001, 0.00002 Bob Smith Sue Jones @@giveRaise: 0.00001, 0.00001 110000.0 @@lastName: 0.00001, 0.00001 @@lastName: 0.00001, 0.00002 Smith Jones ---------------------------------------0.00002 0.00001 0.00002
“Optional” Language Features
1082 | Chapter 39: Metaclasses
www.it-ebooks.info
Example: Applying Decorators to Methods | 1083
www.it-ebooks.info
Chapter Summary
class
Test Your Knowledge: Quiz
1084 | Chapter 39: Metaclasses
www.it-ebooks.info
Test Your Knowledge: Answers type
type class
__new__
__init__
class C(metaclass=M) =M class metaclass
class __metaclass__
class
class
Test Your Knowledge: Answers | 1085
www.it-ebooks.info
www.it-ebooks.info
PART IX
Appendixes
www.it-ebooks.info
www.it-ebooks.info
APPENDIX A
Installation and Configuration
Installing the Python Interpreter
Is Python Already Present?
python python
1089
www.it-ebooks.info
Where to Get Python
1090 | Appendix A: Installation and Configuration
www.it-ebooks.info
Installation Steps
tkinter Tkinter
tkinter
config make Installing the Python Interpreter | 1091
www.it-ebooks.info
The Python MSI Installer on Windows Vista
cd C:\user\downloads msiexec /i python-2.5.1.msi
1092 | Appendix A: Installation and Configuration
www.it-ebooks.info
Configuring Python
Python Environment Variables
Variable
Role
PATH (or path)
System shell search path (for finding “python”)
PYTHONPATH
Python module search path (for imports)
PYTHONSTARTUP
Path to Python interactive startup file
TCL_LIBRARY, TK_LIBRARY
GUI extension variables (tkinter)
PATH PATH
cd C:\Python30
PATH C:\Python30\python PATH
python
Configuring Python | 1093
www.it-ebooks.info
PYTHONPATH PYTHONPATH PYTHONPATH
PATH
sys.path
PYTHONPATH PYTHONSTARTUP PYTHONSTARTUP
tkinter tkinter
Tkinter PYTHONPATH tkinter
Getting tkinter (and IDLE) GUI Support on Linux tkinter tkinter
Tkinter
yum tkinter
1094 | Appendix A: Installation and Configuration
tkinter
www.it-ebooks.info
How to Set Configuration Options
import spam
Unix/Linux shell variables
setenv PYTHONPATH /usr/home/pycode/utilities:/usr/lib/pycode/package1
export PYTHONPATH="/usr/home/pycode/utilities:/usr/lib/pycode/package1"
DOS variables (Windows)
set PYTHONPATH=c:\pycode\utilities;d:\pycode\package1
Windows environment variable GUI PYTHONPATH
Configuring Python | 1095
www.it-ebooks.info
PYTHONPATH set
Windows registry
regedit
Path files PYTHONPATH c:\pycode\utilities d:\pycode\package1
sys.path
1096 | Appendix A: Installation and Configuration
www.it-ebooks.info
Python Command-Line Options
python [-bBdEhiOsSuvVWx?] [-c command | -m module-name | script | - ] [args]
script
args
sys.argv import sys print(sys.argv)
python a b –c
main.py sys.argv
sys.argv c:\Python30> python main.py a b –c ['main.py', 'a', 'b', '-c']
-c – c:\Python30> python -c "print(2 ** 100)" 1267650600228229401496703205376 c:\Python30> python -c "import main" ['-c'] c:\Python30> python - < main.py a b –c ['-', 'a', 'b', '-c'] c:\Python30> python - a b -c < main.py ['-', 'a', 'b', '-c']
–m sys.path
__main__
c:\Python30> python -m main a b –c ['c:\\Python30\\main.py', 'a', 'b', '-c']
–m pdb
profile
Configuring Python | 1097
www.it-ebooks.info
profile
execfile
pdb
io c:\Python30> python -m pdb main.py a b -c --Return-> c:\python30\lib\io.py(762)closed()->False -> return self.raw.closed (Pdb) c c:\Python30> C:\python26\python -m pdb main.py a b -c > c:\python30\main.py(1)() -> import sys (Pdb) c c:\Python30> python -m profile main.py a b -c c:\Python30> python -m cProfile main.py a b -c
-u
-O –i
c:\Python30> python –u main.py a b -c
−3 -Q -t
c:\Python30> python -?
getopt
optparse
For More Help
1098 | Appendix A: Installation and Configuration
www.it-ebooks.info
For More Help | 1099
www.it-ebooks.info
www.it-ebooks.info
APPENDIX B
Solutions to End-of-Part Exercises
Part I, Getting Started
% python ...copyright information lines... >>> "Hello World!" 'Hello World!' >>>
print('Hello module world!') % python module1.py Hello module world!
% python >>> import module1 Hello module world! >>>
1101
www.it-ebooks.info
#! #! #!/usr/local/bin/python print('Hello module world!') % chmod +x module1.py
(or #!/usr/bin/env python)
% module1.py Hello module world!
try
% python >>> 2 ** 500 32733906078961418700131896968275991522166420460430647894832913680961337964046745 54883270092325904157150886684127560071009217256545885393053328527589376 >>> >>> 1 / 0 Traceback (most recent call last): File "", line 1, in ZeroDivisionError: int division or modulo by zero >>> >>> spam Traceback (most recent call last): File "", line 1, in NameError: name 'spam' is not defined
L = [1, 2] L.append(L)
[1, 2, [1, 2, [1, 2, [1, 2
1102 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
[[...]]
L append append L
L
Part II, Types and Operations
; ; Part II, Types and Operations | 1103
www.it-ebooks.info
/ list
>>> 2 ** 16 65536 >>> 2 / 5, 2 / 5.0 (0.40000000000000002, 0.40000000000000002)
>>> "spam" + "eggs" 'spameggs' >>> S = "ham" >>> "eggs " + S 'eggs ham' >>> S * 5 'hamhamhamhamham' >>> S[:0] '' >>> "green %s and %s" % ("eggs", S) 'green eggs and ham' >>> 'green {0} and {1}'.format('eggs', S) 'green eggs and ham'
>>> ('x',)[0] 'x' >>> ('x', 'y')[1] 'y'
>>> L = [1,2,3] + [4,5,6] >>> L, L[:], L[:0], L[-2], L[-2:] ([1, 2, 3, 4, 5, 6], [1, 2, 3, 4, 5, 6], [], 5, [5, 6]) >>> ([1,2,3]+[4,5,6])[2:4] [3, 4] >>> [L[2], L[3]] [3, 4] >>> L.reverse(); L [6, 5, 4, 3, 2, 1] >>> L.sort(); L [1, 2, 3, 4, 5, 6] >>> L.index(4) 3
>>> {'a':1, 'b':2}['b']
1104 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
2 >>> >>> >>> 1 >>>
D = {'x':1, 'y':2, 'z':3} D['w'] = 0 D['x'] + D['w'] D[(1,2,3)] = 4
>>> D {'w': 0, 'z': 3, 'y': 2, (1, 2, 3): 4, 'x': 1} >>> list(D.keys()), list(D.values()), (1,2,3) in D (['w', 'z', 'y', (1, 2, 3), 'x'], [0, 3, 2, 4, 1], True)
>>> [[]], ["",[],(),{},None] ([[]], ['', [], (), {}, None])
L[4] L[-1000:100]
L[3:1]
[ ] L[3:1]
L[3:3]
3
L[3:1:-1] >>> L = [1, 2, 3, 4] >>> L[4] Traceback (innermost last): File "", line 1, in ? IndexError: list index out of range >>> L[-1000:100] [1, 2, 3, 4] >>> L[3:1] [] >>> L [1, 2, 3, 4] >>> L[3:1] = ['?'] >>> L [1, 2, 3, '?', 4]
Part II, Types and Operations | 1105
www.it-ebooks.info
>>> L = [1,2,3,4] >>> L[2] = [] >>> L [1, 2, [], 4] >>> L[2:3] = [] >>> L [1, 2, 4] >>> del L[0] >>> L [2, 4] >>> del L[1:] >>> L [2] >>> L[1:2] = 1 Traceback (innermost last): File "", line 1, in ? TypeError: illegal argument type for built-in operation
X
Y =
>>> X = 'spam' >>> Y = 'eggs' >>> X, Y = Y, X >>> X 'eggs' >>> Y 'spam'
>>> >>> >>> >>> >>> {1:
D = {} D[1] = 'a' D[2] = 'b' D[(1, 2, 3)] = 'c' D 'a', 2: 'b', (1, 2, 3): 'c'}
D['d'] D['d']='spam'
1106 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
>>> D = {'a':1, 'b':2, 'c':3} >>> D['a'] 1 >>> D['d'] Traceback (innermost last): File "", line 1, in ? KeyError: d >>> D['d'] = 4 >>> D {'b': 2, 'd': 4, 'a': 1, 'c': 3} >>> >>> L = [0, 1] >>> L[2] Traceback (innermost last): File "", line 1, in ? IndexError: list index out of range >>> L[2] = 3 Traceback (innermost last): File "", line 1, in ? IndexError: list assignment index out of range
+
+
+
+ append
keys append
>>> "x" + 1 Traceback (innermost last): File "", line 1, in ? TypeError: illegal argument type for built-in operation >>> >>> {} + {} Traceback (innermost last): File "", line 1, in ? TypeError: bad operand type(s) for + >>> >>> [].append(9) >>> "".append('s') Traceback (innermost last): File "", line 1, in ? AttributeError: attribute-less object >>> >>> list({}.keys()) [] >>> [].keys() Traceback (innermost last): File "", line 1, in ? AttributeError: keys
Part II, Types and Operations | 1107
www.it-ebooks.info
>>> >>> [][:] [] >>> ""[:] ''
S[0][0][0][0][0]
>>> >>> 's' >>> >>> 's'
S = "spam" S[0][0][0][0][0] L = ['s', 'p'] L[0][0][0]
>>> S = "spam" >>> S = S[0] + 'l' + S[2:] >>> S 'slam' >>> S = S[0] + 'l' + S[2] + S[3] >>> S 'slam'
bytearray
>>> me = {'name':('John', 'Q', 'Doe'), 'age':'?', 'job':'engineer'} >>> me['job'] 'engineer' >>> me['name'][2] 'Doe'
ls dir file = open('myfile.txt', 'w') file.write('Hello file world!\n') file.close() file = open('myfile.txt') print(file.read()) % python maker.py % python reader.py Hello file world!
1108 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
% ls -l myfile.txt -rwxrwxrwa 1 0
0
19 Apr 13 16:33 myfile.txt
Part III, Statements and Syntax
>>> S = 'spam' >>> for c in S: ... print(ord(c)) ... 115 112 97 109 >>> x = 0 >>> for c in S: x += ord(c) ... >>> x 433 >>> x = [] >>> for c in S: x.append(ord(c)) ... >>> x [115, 112, 97, 109] >>> list(map(ord, S)) [115, 112, 97, 109]
\a
keys
sort
sort
None keys
for key in D: sorted >>> D = {'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7} >>> D {'f': 6, 'c': 3, 'a': 1, 'g': 7, 'e': 5, 'd': 4, 'b': 2} >>> >>> keys = list(D.keys())
Part III, Statements and Syntax | 1109
www.it-ebooks.info
>>> keys.sort() >>> for key in keys: ... print(key, '=>', D[key]) ... a => 1 b => 2 c => 3 d => 4 e => 5 f => 6 g => 7 >>> for key in sorted(D): ... print(key, '=>', D[key])
e 2 ** X
L = [1, 2, 4, 8, 16, 32, 64] X = 5 i = 0 while i < len(L): if 2 ** X == L[i]: print('at index', i) break i += 1 else: print(X, 'not found')
L = [1, 2, 4, 8, 16, 32, 64] X = 5 for p in L: if (2 ** X) == p: print((2 ** X), 'was found at', L.index(p)) break else: print(X, 'not found')
L = [1, 2, 4, 8, 16, 32, 64] X = 5 if (2 ** X) in L: print((2 ** X), 'was found at', L.index(2 ** X))
1110 | Appendix B: Solutions to End-of-Part Exercises
a
b
www.it-ebooks.info
else: print(X, 'not found')
X = 5 L = [] for i in range(7): L.append(2 ** i) print(L) if (2 ** X) in L: print((2 ** X), 'was found at', L.index(2 ** X)) else: print(X, 'not found') # X = 5 L = list(map(lambda x: 2**x, range(7))) print(L) if (2 ** X) in L: print((2 ** X), 'was found at', L.index(2 ** X)) else: print(X, 'not found')
Part IV, Functions print
% python >>> def func(x): print(x) ... >>> func("spam") spam >>> func(42) 42 >>> func([1, 2, 3]) [1, 2, 3] >>> func({'food': 'spam'}) {'food': 'spam'}
print
Part IV, Functions | 1111
www.it-ebooks.info
def adder(x, y): return x + y print(adder(2, 3)) print(adder('spam', 'eggs')) print(adder(['a', 'b'], ['c', 'd'])) % python mod.py 5 spameggs ['a', 'b', 'c', 'd']
adder
min +
def adder1(*args): print('adder1', end=' ') if type(args[0]) == type(0): sum = 0 else: sum = args[0][:0] for arg in args: sum = sum + arg return sum def adder2(*args): print('adder2', end=' ') sum = args[0] for next in args[1:]: sum += next return sum for func in (adder1, adder2): print(func(2, 3, 4)) print(func('spam', 'eggs', 'toast')) print(func(['a', 'b'], ['c', 'd'], ['e', 'f'])) % python adders.py adder1 9 adder1 spameggstoast adder1 ['a', 'b', 'c', 'd', 'e', 'f'] adder2 9 adder2 spameggstoast adder2 ['a', 'b', 'c', 'd', 'e', 'f']
1112 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
args.values()
**args for x in args.keys(): use args[x] *args
def adder(good=1, bad=2, ugly=3): return good + bad + ugly print(adder()) print(adder(5)) print(adder(5, 6)) print(adder(5, 6, 7)) print(adder(ugly=7, good=6, bad=5)) % python mod.py 6 10 14 18 18
def adder1(*args): tot = args[0] for arg in args[1:]: tot += arg return tot def adder2(**args): argskeys = list(args.keys()) tot = args[argskeys[0]] for key in argskeys[1:]: tot += args[key] return tot def adder3(**args): args = list(args.values()) tot = args[0] for arg in args[1:]: tot += arg return tot def adder4(**args): return adder1(*args.values()) print(adder1(1, 2, 3), adder1('aa', 'bb', 'cc')) print(adder2(a=1, b=2, c=3), adder2(a='aa', b='bb', c='cc')) print(adder3(a=1, b=2, c=3), adder3(a='aa', b='bb', c='cc')) print(adder4(a=1, b=2, c=3), adder4(a='aa', b='bb', c='cc'))
Part IV, Functions | 1113
www.it-ebooks.info
D.copy()
D1.update(D2) X[:] e = d d
def copyDict(old): new = {} for key in old.keys(): new[key] = old[key] return new def addDict(d1, d2): new = {} for key in d1.keys(): new[key] = d1[key] for key in d2.keys(): new[key] = d2[key] return new % python >>> from dicts import * >>> d = {1: 1, 2: 2} >>> e = copyDict(d) >>> d[2] = '?' >>> d {1: 1, 2: '?'} >>> e {1: 1, 2: 2} >>> >>> >>> >>> {1:
x = {1: 1} y = {2: 2} z = addDict(x, y) z 1, 2: 2}
def f1(a, b): print(a, b) def f2(a, *b): print(a, b) def f3(a, **b): print(a, b) def f4(a, *b, **c): print(a, b, c) def f5(a, b=2, c=3): print(a, b, c)
1114 | Appendix B: Solutions to End-of-Part Exercises
e
www.it-ebooks.info
def f6(a, b=2, *c): print(a, b, c) % python >>> f1(1, 2) 1 2 >>> f1(b=2, a=1) 1 2 >>> f2(1, 2, 3) 1 (2, 3) >>> f3(1, x=2, y=3) 1 {'x': 2, 'y': 3} >>> f4(1, 2, 3, x=2, y=3) 1 (2, 3) {'x': 2, 'y': 3} >>> 1 2 >>> 1 4
f5(1) 3 f5(1, 4) 3
>>> 1 2 >>> 1 3
f6(1) () f6(1, 3, 4) (4,)
if 0
1
/
//
/ from //
/
#from __future__ import division def prime(y): if y 1: if y % x == 0: print(y, 'has factor', x) break x -= 1 else: print(y, 'is prime') prime(13); prime(15); prime(3); prime(1);
prime(13.0) prime(15.0) prime(2) prime(-3)
Part IV, Functions | 1115
www.it-ebooks.info
// % python primes.py 13 is prime 13.0 is prime 15 has factor 5 15.0 has factor 5.0 3 is prime 2 is prime 1 not prime -3 not prime
for while time def timer(reps, func, *args): import time start = time.clock() for i in range(reps): func(*args) return time.clock() - start
>>> values = [2, 4, 9, 16, 25] >>> import math >>> res = [] >>> for x in values: res.append(math.sqrt(x)) ... >>> res [1.4142135623730951, 2.0, 3.0, 4.0, 5.0] >>> list(map(math.sqrt, values)) [1.4142135623730951, 2.0, 3.0, 4.0, 5.0] >>> [math.sqrt(x) for x in values] [1.4142135623730951, 2.0, 3.0, 4.0, 5.0]
...same as listed in Chapter 20...
1116 | Appendix B: Solutions to End-of-Part Exercises
range(y, 1, −1)
www.it-ebooks.info
import sys, mytimer reps = 10000 repslist = range(reps) from math import sqrt def mathMod(): for i in repslist: res = sqrt(i) return res def powCall(): for i in repslist: res = pow(i, .5) return res def powExpr(): for i in repslist: res = i ** .5 return res print(sys.version) for tester in (mytimer.timer, mytimer.best): print('' % tester.__name__) for test in (mathMod, powCall, powExpr): elapsed, result = tester(test) print ('-'*35) print ('%s: %.5f => %s' % (test.__name__, elapsed, result))
math
**
pow
c:\misc> c:\python30\python timesqrt.py 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)] ----------------------------------mathMod: 5.33906 => 99.994999875 ----------------------------------powCall: 7.29689 => 99.994999875 ----------------------------------powExpr: 5.95770 => 99.994999875 ----------------------------------mathMod: 0.00497 => 99.994999875 ----------------------------------powCall: 0.00671 => 99.994999875 ----------------------------------powExpr: 0.00540 => 99.994999875 c:\misc> c:\python26\python timesqrt.py
Part IV, Functions | 1117
www.it-ebooks.info
2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] ----------------------------------mathMod: 2.61226 => 99.994999875 ----------------------------------powCall: 4.33705 => 99.994999875 ----------------------------------powExpr: 3.12502 => 99.994999875 ----------------------------------mathMod: 0.00236 => 99.994999875 ----------------------------------powCall: 0.00402 => 99.994999875 ----------------------------------powExpr: 0.00287 => 99.994999875
for
c:\misc> c:\python30\python >>> >>> def dictcomp(I): ... return {i: i for i in range(I)} ... >>> def dictloop(I): ... new = {} ... for i in range(I): new[i] = i ... return new ... >>> dictcomp(10) {0: 0, 1: 1, 2: 2, 3: 3, 4: 4, 5: 5, 6: 6, 7: 7, 8: 8, 9: 9} >>> dictloop(10) {0: 0, 1: 1, 2: 2, 3: 3, 4: 4, 5: 5, 6: 6, 7: 7, 8: 8, 9: 9} >>> >>> from mytimer import best, timer >>> best(dictcomp, 10000)[0] 0.0013519874732672577 >>> best(dictloop, 10000)[0] 0.001132965223233029 >>> >>> best(dictcomp, 100000)[0] 0.01816089754424155 >>> best(dictloop, 100000)[0] 0.01643484018219965 >>> >>> best(dictcomp, 1000000)[0] 0.18685105229855026 >>> best(dictloop, 1000000)[0]
1118 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
0.1769041177020938 >>> >>> timer(dictcomp, 1000000, _reps=50)[0] 10.692516087938543 >>> timer(dictloop, 1000000, _reps=50)[0] 10.197276050447755
Part V, Modules
len def countLines(name): file = open(name) return len(file.readlines()) def countChars(name): return len(open(name).read()) def test(name): return countLines(name), countChars(name) % python >>> import mymod >>> mymod.test('mymod.py') (10, 291)
def countLines(name): tot = 0 for line in open(name): tot += 1 return tot def countChars(name): tot = 0 for line in open(name): tot += len(line) return tot
wc \r\n \n 'rb'
Part V, Modules | 1119
www.it-ebooks.info
seek fseek seek seek file.seek(0)
read
def countLines(file): file.seek(0) return len(file.readlines()) def countChars(file): file.seek(0) return len(file.read()) def test(name): file = open(name) return countLines(file), countChars(file) >>> import mymod2 >>> mymod2.test("mymod2.py") (11, 392)
from from *
from *
*
countChars
% python >>> from mymod import * >>> countChars("mymod.py") 291
__main__ def countLines(name): file = open(name) return len(file.readlines()) def countChars(name): return len(open(name).read()) def test(name): return countLines(name), countChars(name) if __name__ == '__main__': print(test('mymod.py')) % python mymod.py (13, 346)
sys.argv
1120 | Appendix B: Solutions to End-of-Part Exercises
input
www.it-ebooks.info
if __name__ == '__main__': print(test(input('Enter file name:')) if __name__ == '__main__': import sys print(test(sys.argv[1]))
from mymod import countLines, countChars print(countLines('mymod.py'), countChars('mymod.py')) % python myclient.py 13 346
mymod myclient from mymod def
myclient
import myclient myclient.countLines(...) from myclient import countChars countChars(...)
myclient
import mymod
from myclient
import myclient myclient.mymod.countLines(...) from myclient import mymod mymod.countChars(...)
somename mod1.somename collector.somename
__main__.somename somename
somename = 42 from mod1 import * from mod2 import * from mod3 import * >>> from collector import somename
mv
vi
move
edit
Part V, Modules | 1121
www.it-ebooks.info
print C:\python30> mkdir mypkg C:\Python30> move mymod.py mypkg\mymod.py C:\Python30> edit mypkg\__init__.py ...coded a print statement... C:\Python30> python >>> import mypkg.mymod initializing mypkg >>> mypkg.mymod.countLines('mypkg\mymod.py') 13 >>> from mypkg.mymod import countChars >>> countChars('mypkg\mymod.py') 346
recur2 recur1 recur1 recur2 recur2 import from recur2 recur2
import
from
recur2
recur2 recur2 recur1 recur1 from
from
Y
recur1
recur1 recur2
recur1
Part VI, Classes and OOP
__add__ add class Adder: def add(self, x, y):
1122 | Appendix B: Solutions to End-of-Part Exercises
recur2
www.it-ebooks.info
print('not implemented!') def __init__(self, start=[]): self.data = start def __add__(self, other): return self.add(self.data, other) class ListAdder(Adder): def add(self, x, y): return x + y class DictAdder(Adder): def add(self, x, y): new = {} for k in x.keys(): new[k] = x[k] for k in y.keys(): new[k] = y[k] return new % python >>> from adder import * >>> x = Adder() >>> x.add(1, 2) not implemented! >>> x = ListAdder() >>> x.add([1], [2]) [1, 2] >>> x = DictAdder() >>> x.add({1:1}, {2:2}) {1: 1, 2: 2} >>> x = Adder([1]) >>> x + [2] not implemented! >>> >>> x = ListAdder([1]) >>> x + [2] [1, 2] >>> [2] + x Traceback (innermost last): File "", line 1, in ? TypeError: __add__ nor __radd__ defined for these operands
+
__radd__
add class Adder: def __init__(self, start=[]): self.data = start def __add__(self, other): return self.add(other) def add(self, y):
Part VI, Classes and OOP | 1123
www.it-ebooks.info
print('not implemented!') class ListAdder(Adder): def add(self, y): return self.data + y class DictAdder(Adder): def add(self, y): pass x = ListAdder([1, 2, 3]) y = x + [4, 5, 6] print(y)
add __add__
__getattr__
class MyList: def __init__(self, start): #self.wrapped = start[:] self.wrapped = [] for x in start: self.wrapped.append(x) def __add__(self, other): return MyList(self.wrapped + other) def __mul__(self, time): return MyList(self.wrapped * time) def __getitem__(self, offset): return self.wrapped[offset] def __len__(self): return len(self.wrapped) def __getslice__(self, low, high): return MyList(self.wrapped[low:high]) def append(self, node): self.wrapped.append(node) def __getattr__(self, name): return getattr(self.wrapped, name) def __repr__(self): return repr(self.wrapped) if __name__ == '__main__': x = MyList('spam') print(x) print(x[2]) print(x[1:])
1124 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
print(x + ['eggs']) print(x * 3) x.append('a') x.sort() for c in x: print(c, end=' ') % python mylist.py ['s', 'p', 'a', 'm'] a ['p', 'a', 'm'] ['s', 'p', 'a', 'm', 'eggs'] ['s', 'p', 'a', 'm', 's', 'p', 'a', 'm', 's', 'p', 'a', 'm'] a a m p s
append MyList
from mylist import MyList class MyListSub(MyList): calls = 0 def __init__(self, start): self.adds = 0 MyList.__init__(self, start) def __add__(self, other): MyListSub.calls += 1 self.adds += 1 return MyList.__add__(self, other) def stats(self): return self.calls, self.adds if __name__ == '__main__': x = MyListSub('spam') y = MyListSub('foo') print(x[2]) print(x[1:]) print(x + ['eggs']) print(x + ['toast']) print(y + ['bar']) print(x.stats()) % python mysub.py
Part VI, Classes and OOP | 1125
www.it-ebooks.info
a ['p', 'a', ['s', 'p', ['s', 'p', ['f', 'o', (3, 2)
'm'] 'a', 'm', 'eggs'] 'a', 'm', 'toast'] 'o', 'bar']
__getattr__ __getattr__ __X__ >>> class Meta: ... def __getattr__(self, name): ... print('get', name) ... def __setattr__(self, name, value): ... print('set', name, value) ... >>> x = Meta() >>> x.append get append >>> x.spam = "pork" set spam pork >>> >>> x + 2 get __coerce__ Traceback (innermost last): File "", line 1, in ? TypeError: call of non-function >>> >>> x[1] get __getitem__ Traceback (innermost last): File "", line 1, in ? TypeError: call of non-function >>> x[1:5] get __len__ Traceback (innermost last): File "", line 1, in ? TypeError: call of non-function
% python >>> from setwrapper import Set >>> x = Set([1, 2, 3, 4]) >>> y = Set([3, 4, 5]) >>> x & y Set:[3, 4] >>> x | y
1126 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
Set:[1, 2, 3, 4, 5] >>> z = Set("hello") >>> z[0], z[-1] ('h', 'o') >>> ... h e >>> (4,
for c in z: print(c, end=' ') l o len(z), z Set:['h', 'e', 'l', 'o'])
>>> z & "mello", z | "mello" (Set:['e', 'l', 'o'], Set:['h', 'e', 'l', 'o', 'm'])
from setwrapper import Set class MultiSet(Set): """ Inherits all Set names, but extends intersect and union to support multiple operands; note that "self" is still the first argument (stored in the *args argument now); also note that the inherited & and | operators call the new methods here with 2 arguments, but processing more than 2 requires a method call, not an expression: """ def intersect(self, *others): res = [] for x in self: for other in others: if x not in other: break else: res.append(x) return Set(res) def union(*args): res = [] for seq in args: for x in seq: if not x in res: res.append(x) return Set(res)
& intersect
intersect &
MultiSet setwrapper.Set
Set multiset
Part VI, Classes and OOP | 1127
www.it-ebooks.info
>>> >>> >>> >>>
from multiset import * x = MultiSet([1,2,3,4]) y = MultiSet([3,4,5]) z = MultiSet([0,1,2])
>>> x & y, x | y (Set:[3, 4], Set:[1, 2, 3, 4, 5]) >>> x.intersect(y, z) Set:[] >>> x.union(y, z) Set:[1, 2, 3, 4, 5, 0] >>> x.intersect([1,2,3], [2,3,4], [1,2,3]) Set:[2, 3] >>> x.union(range(10)) Set:[1, 2, 3, 4, 0, 5, 6, 7, 8, 9]
dir class ListInstance: def __str__(self): return '' % ( self.__class__.__name__, self.__supers(), id(self), self.__attrnames()) ) def __attrnames(self): ...unchanged... def __supers(self): names = [] for super in self.__class__.__bases__: names.append(super.__name__) return ', '.join(names) C:\misc> python testmixin.py
class Lunch: def __init__(self): self.cust = Customer() self.empl = Employee() def order(self, foodName): self.cust.placeOrder(foodName, self.empl) def result(self): self.cust.printFood()
1128 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
class Customer: def __init__(self): self.food = None def placeOrder(self, foodName, employee): self.food = employee.takeOrder(foodName) def printFood(self): print(self.food.name) class Employee: def takeOrder(self, foodName): return Food(foodName) class Food: def __init__(self, name): self.name = name if __name__ == '__main__': x = Lunch() x.order('burritos') x.result() x.order('pizza') x.result() % python lunch.py burritos pizza
self.speak speak
class Animal: def reply(self): def speak(self):
self.speak() print('spam')
class Mammal(Animal): def speak(self):
print('huh?')
class Cat(Mammal): def speak(self):
print('meow')
class Dog(Mammal): def speak(self):
print('bark')
Animal
class Primate(Mammal): def speak(self): print('Hello world!') class Hacker(Primate): pass
Part VI, Classes and OOP | 1129
www.it-ebooks.info
line
Actor
self.name
self
self.says()
class Actor: def line(self): print(self.name + ':', repr(self.says())) class Customer(Actor): name = 'customer' def says(self): return "that's one ex-bird!" class Clerk(Actor): name = 'clerk' def says(self): return "no it isn't..." class Parrot(Actor): name = 'parrot' def says(self): return None class Scene: def __init__(self): self.clerk = Clerk() self.customer = Customer() self.subject = Parrot() def action(self): self.customer.line() self.clerk.line() self.subject.line()
Part VII, Exceptions and Tools try except
oops oops
KeyError
IndexError
try KeyError Error __builtin__ def oops(): raise IndexError() def doomed(): try: oops() except IndexError:
1130 | Appendix B: Solutions to End-of-Part Exercises
dir
Index builtins
www.it-ebooks.info
print('caught an index error!') else: print('no error caught...') if __name__ == '__main__': doomed() % python oops.py caught an index error!
class MyError(Exception): pass def oops(): raise MyError('Spam!') def doomed(): try: oops() except IndexError: print('caught an index error!') except MyError as data: print('caught error:', MyError, data) else: print('no error caught...') if __name__ == '__main__': doomed() % python oops.py caught error: Spam!
__init__
__repr__
Spam! __str__
Exception
import sys, traceback def safe(entry, *args): try: entry(*args) except: traceback.print_exc() print('Got', sys.exc_info()[0], sys.exc_info()[1]) import oops safe(oops.oops) % python safe2.py
Part VII, Exceptions and Tools | 1131
www.it-ebooks.info
Traceback (innermost last): File "safe2.py", line 5, in safe entry(*args) File "oops.py", line 4, in oops raise MyError, 'world' hello: world Got hello world
import os, glob dirname = r'C:\Python30\Lib' allsizes = [] allpy = glob.glob(dirname + os.sep + '*.py') for filename in allpy: filesize = os.path.getsize(filename) allsizes.append((filesize, filename)) allsizes.sort() print(allsizes[:2]) print(allsizes[-2:])
import sys, os, pprint if sys.platform[:3] == 'win': dirname = r'C:\Python30\Lib' else: dirname = '/usr/lib/python' allsizes = [] for (thisDir, subsHere, filesHere) in os.walk(dirname): for filename in filesHere: if filename.endswith('.py'): fullname = os.path.join(thisDir, filename) fullsize = os.path.getsize(fullname) allsizes.append((fullsize, fullname)) allsizes.sort() pprint.pprint(allsizes[:2]) pprint.pprint(allsizes[-2:])
import sys, os, pprint visited = {} allsizes = [] for srcdir in sys.path: for (thisDir, subsHere, filesHere) in os.walk(srcdir): thisDir = os.path.normpath(thisDir)
1132 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
if thisDir.upper() in visited: continue else: visited[thisDir.upper()] = True for filename in filesHere: if filename.endswith('.py'): pypath = os.path.join(thisDir, filename) try: pysize = os.path.getsize(pypath) except: print('skipping', pypath) allsizes.append((pysize, pypath)) allsizes.sort() pprint.pprint(allsizes[:3]) pprint.pprint(allsizes[-3:])
filename = 'data.txt' sums = {} for line in open(filename): cols = line.split(',') nums = [int(col) for col in cols] for (ix, num) in enumerate(nums): sums[ix] = sums.get(ix, 0) + num for key in sorted(sums): print(key, '=', sums[key])
import sys filename = sys.argv[1] numcols = int(sys.argv[2]) totals = [0] * numcols for line in open(filename): cols = line.split(',') nums = [int(x) for x in cols] totals = [(x + y) for (x, y) in zip(totals, nums)] print(totals)
import os testscripts = [dict(script='test1.py', args=''), dict(script='test2.py', args='spam')] for testcase in testscripts:
Part VII, Exceptions and Tools | 1133
www.it-ebooks.info
commandline = '%(script)s %(args)s' % testcase output = os.popen(commandline).read() result = testcase['script'] + '.result' if not os.path.exists(result): open(result, 'w').write(output) print('Created:', result) else: priorresult = open(result).read() if output != priorresult: print('FAILED:', testcase['script']) print(output) else: print('Passed:', testcase['script'])
from tkinter import * import random fontsize = 25 colors = ['red', 'green', 'blue', 'yellow', 'orange', 'white', 'cyan', 'purple'] def reply(text): print(text) popup = Toplevel() color = random.choice(colors) Label(popup, text='Popup', bg='black', fg=color).pack() L.config(fg=color) def timer(): L.config(fg=random.choice(colors)) win.after(250, timer) def grow(): global fontsize fontsize += 5 L.config(font=('arial', fontsize, 'italic')) win.after(100, grow) win = Tk() L = Label(win, text='Spam', font=('arial', fontsize, 'italic'), fg='yellow', bg='navy', relief=RAISED) L.pack(side=TOP, expand=YES, fill=BOTH) Button(win, text='press', command=(lambda: reply('red'))).pack(side=BOTTOM,fill=X) Button(win, text='timer', command=timer).pack(side=BOTTOM, fill=X) Button(win, text='grow', command=grow).pack(side=BOTTOM, fill=X) win.mainloop()
from tkinter import * import random
1134 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
class MyGui: """ A GUI with buttons that change color and make the label grow """ colors = ['blue', 'green', 'orange', 'red', 'brown', 'yellow'] def __init__(self, parent, title='popup'): parent.title(title) self.growing = False self.fontsize = 10 self.lab = Label(parent, text='Gui1', fg='white', bg='navy') self.lab.pack(expand=YES, fill=BOTH) Button(parent, text='Spam', command=self.reply).pack(side=LEFT) Button(parent, text='Grow', command=self.grow).pack(side=LEFT) Button(parent, text='Stop', command=self.stop).pack(side=LEFT) def reply(self): "change the button's color at random on Spam presses" self.fontsize += 5 color = random.choice(self.colors) self.lab.config(bg=color, font=('courier', self.fontsize, 'bold italic')) def grow(self): "start making the label grow on Grow presses" self.growing = True self.grower() def grower(self): if self.growing: self.fontsize += 5 self.lab.config(font=('courier', self.fontsize, 'bold')) self.lab.after(500, self.grower) def stop(self): "stop the button growing on Stop presses" self.growing = False class MySubGui(MyGui): colors = ['black', 'purple'] MyGui(Tk(), 'main') MyGui(Toplevel()) MySubGui(Toplevel()) mainloop()
""" scan pop email box, fetching just headers, allowing deletions without downloading the complete message """ import poplib, getpass, sys
Part VII, Exceptions and Tools | 1135
www.it-ebooks.info
mailserver = 'your pop email server name here' mailuser = 'your pop email user name here' mailpasswd = getpass.getpass('Password for %s?' % mailserver) print('Connecting...') server = poplib.POP3(mailserver) server.user(mailuser) server.pass_(mailpasswd) try:
print(server.getwelcome()) msgCount, mboxSize = server.stat() print('There are', msgCount, 'mail messages, size ', mboxSize) msginfo = server.list() print(msginfo) for i in range(msgCount): msgnum = i+1 msgsize = msginfo[1][i].split()[1] resp, hdrlines, octets = server.top(msgnum, 0) print('-'*80) print('[%d: octets=%d, size=%s]' % (msgnum, octets, msgsize)) for line in hdrlines: print(line) if input('Print?') in ['y', 'Y']: for line in server.retr(msgnum)[1]: print(line) if input('Delete?') in ['y', 'Y']: print('deleting') server.dele(msgnum) else: print('skipping')
finally: server.quit() input('Bye.')
#!/usr/bin/python import cgi form = cgi.FieldStorage() print("Content-type: text/html\n") print("") print("Reply Page") print("") if not 'user' in form: print("Who are you?") else: print("Hello %s!" % cgi.escape(form['user'].value)) print("")
from MySQLdb import Connect
1136 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
conn = Connect(host='localhost', user='root', passwd='darling') curs = conn.cursor() try: curs.execute('drop database testpeopledb') except: pass curs.execute('create database testpeopledb') curs.execute('use testpeopledb') curs.execute('create table people (name char(30), job char(10), pay int(4))') curs.execute('insert people values (%s, %s, %s)', ('Bob', 'dev', 50000)) curs.execute('insert people values (%s, %s, %s)', ('Sue', 'dev', 60000)) curs.execute('insert people values (%s, %s, %s)', ('Ann', 'mgr', 40000)) curs.execute('select * from people') for row in curs.fetchall(): print(row) curs.execute('select * from people where name = %s', ('Bob',)) print(curs.description) colnames = [desc[0] for desc in curs.description] while True: print('-' * 30) row = curs.fetchone() if not row: break for (name, value) in zip(colnames, row): print('%s => %s' % (name, value)) conn.commit()
# see also Chapter 27 shelve and Chapter 30 pickle examples rec1 = {'name': {'first': 'Bob', 'last': 'Smith'}, 'job': ['dev', 'mgr'], 'age': 40.5} rec2 = {'name': {'first': 'Sue', 'last': 'Jones'}, 'job': ['mgr'], 'age': 35.0} import shelve db = shelve.open('dbfile') db['bob'] = rec1 db['sue'] = rec2 db.close()
import shelve db = shelve.open('dbfile')
Part VII, Exceptions and Tools | 1137
www.it-ebooks.info
for key in db: print(key, '=>', db[key]) bob = db['bob'] bob['age'] += 1 db['bob'] = bob db.close()
1138 | Appendix B: Solutions to End-of-Part Exercises
www.it-ebooks.info
Index
Symbols
1139
www.it-ebooks.info
A
1140 | Index
www.it-ebooks.info
B
C
Index | 1141
www.it-ebooks.info
1142 | Index
www.it-ebooks.info
D
Index | 1143
www.it-ebooks.info
E
1144 | Index
www.it-ebooks.info
F
Index | 1145
www.it-ebooks.info
1146 | Index
www.it-ebooks.info
H
I G
Index | 1147
www.it-ebooks.info
J
K
1148 | Index
www.it-ebooks.info
L
M
Index | 1149
www.it-ebooks.info
1150 | Index
www.it-ebooks.info
N
Index | 1151
www.it-ebooks.info
O
1152 | Index
www.it-ebooks.info
P
Index | 1153
www.it-ebooks.info
1154 | Index
www.it-ebooks.info
Q
R
Index | 1155
www.it-ebooks.info
S
1156 | Index
www.it-ebooks.info
Index | 1157
www.it-ebooks.info
T
1158 | Index
www.it-ebooks.info
U
V
Index | 1159
www.it-ebooks.info
W
X Y
Z
1160 | Index
www.it-ebooks.info
About the Author
Colophon
www.it-ebooks.info
View more...
Comments
