covex_assignment

Problem Set 1 January 7, 2014 1.

(a) Prove that for rectangular matrices B, D, and invertible matrices A, C A(A + BCD)−1 = I − B(C−1 + DA−1 B)−1 DA−1 . (b) Using the above result, prove that (A−1 + BT C−1 B)−1 BT C−1 = ABT (BABT + C)−1 . (c) Find an expression for (A + xyT )−1 in terms of A−1 , x and y for invertible matrix A and (column) vectors x and y.

2. Let λi (A) denote an eigenvalue of A (a) λi (AB) = λi (BA) where A, B are not necessarily square. (b) If A = AT , find the following in terms of λi (A) i. ii. iii. iv.

Tr(Ap ). λi (I + cA). λi (A − cI). λi (A−1 ).

3. The notation A ≻ 0 denotes the fact that A is positive definite. Prove the following results for A ≻ 0: (a) A−1 ≻ 0

(b) [A]ii ≥ 0 for all i, where [A]ii denotes the i-th diagonal entry of A. (c) For any B, rank(BABT ) = rank(B).

(d) If B is full row-rank, then BABT ≻ 0.

(e) If A  0, then for any matrix X, XAXT = 0 ⇒ AX = 0.

4. Prove the following for the operator norm (with a = b, and all vector norms are k.ka ). (a) kIka,a = 1

(b) kAka,a = supkxk=1 kAxka (c) kAka,a = supkxk6=0

kAxk . kxk

(d) Let AT A  0, then kAk22,2 = maxi λi (AT A) (e) kAk1,1 = maxj

P

i |[A]i,j |

(f) kAxk ≤ kAka,a kxk

(g) kABka,a ≤ kAka,a kBka,a 5. Consider X = {x ∈ Rn |Ax = b} for A ∈ Rm×n and b ∈ Rm with b ∈ R(A). Prove that the following two statements are equivalent (i.e., (a) ⇒ (b) and (b) ⇒ (a)) (a) cT x = d for all x ∈ X

(b) There exists v ∈ Rm such that c = AT v and d = bT v. 6. Prove that (a) ku + vk22 = kuk22 + kvk22 if and only if uT v = 0.

(b) 2 < a, b > +2 < x, y >=< a + x, b + y > + < a − x, b − y > √ (c) kxk1 ≤ n kxk2

(d) kxk1 ≥ kxk2 ≥ kxk∞

7. Consider A ∈ Rm×n where m ≥ n and rank of A is n. Suppose there exists B such that BA = I and that AB = BT AT . (a) Let X = AB. Show that i. X2 = X ii. Tr(X) = rank(X) (b) Find an expression for B in terms of A. (c) For any two vectors b ∈ Rn and c ∈ Rm , prove that kAb − ck2 ≥ kABc − ck2 .

EE609

Assignment 2

January 8, 2015

Weight: 1% if submitted by Jan 15, 2014. Assignment will not be graded. No need to submit practice problems.

1 Assignment Problems 1. Show that the set {x ∈ Rn |kx − x1 k2 ≤ kx − x2 k2 } is actually a half space. and express it in canonical form. 2. Show that the set {x ∈ Rn |kxk1 ≤ 2} is a polyhedra and express it as an intersection of half spaces (and hyperplanes, if required). 3. Show that the set {x ∈ R2++ |x1 x2 ≥ 1} is convex and express it as an intersection of infinite number of half-spaces. 4. Is the following set convex: {x ∈ Rn |x ≥ 0, xT y ≤ 1 for all y with kyk2 = 1}? Show that the set {x ∈ Rn |xT y ≤ 1 for all y with y ∈ C} is always convex, even if C is not convex. 5. Consider the two-dimensional positive semidefinite cone S2+ defined as {X = [ xy yz ] |x, y, z ∈ R, X  0}

(1)

Show that it can equivalently be expressed as {x, y, z ∈ R|x ≥ 0, z ≥ 0, xz ≥ y 2 }.

2 Practice Problems 6. Show that a set is affine if and only if its intersection with any line is affine. 7. What is the minimum distance between two parallel halfspaces {x ∈ Rn |aT x ≤ b1 } and {x ∈ Rn |aT x ≥ b2 } (Hint: it depends on where the origin is)? ˜, b, and ˜b, is the following true? 8. Under what conditions on a, a {x ∈ Rn |aT x ≤ b} ⊆ {x ∈ Rn |˜ aT x ≤ ˜b}

(2)

9. For m < n, let b ∈ Rm and A ∈ Rm×n be full row rank. Show that any affine set {x ∈ Rn |Ax = b} can be expressed in the form {Cu + v|u ∈ Rm }. For example, the set {x ∈ R2 |x1 + x2 = 1} can be expressed as {[u 1 − u]T |u ∈ R} 10. What is the affine hull of the set {x ∈ R3 |x21 + x22 = 1, x3 = 1}. 11. Is the following set affine: {x ∈ Rn |kx − x1 k1 ≤ kx − x2 k1 }? 12. Given two vectors y, z ∈ Rn , consider the set S{a,b} = {x ∈ Rn |x = ay + bz}. Show that the following set is a polyhedra, and find its boundaries. [ [ S{a,b} (3) a∈[−1,1] b∈[−1,1]

13. Show that the set {x ∈ Rn |kxk∞ ≤ 1} is a polyhedra and express it as an intersection of half spaces (and hyperplanes, if required).

EE609

Assignment 2, Page 2 of 2

January 8, 2015

14. Show that the set of all doubly stochastic matrices is convex polyhedral in Rn×n . A doubly stochastic matrix is a square matrix with nonnegative entries with the property that the sum of entries in every row and column is exactly 1. 15. What kind of set is C = {x ∈ R2 |p(0) = 1, |p(t)| ≤ 1, for 1 ≤ t ≤ 2} where p(t) = x1 + x2 t? 16. Given θ, consider the set S = {x ∈ Rn |kx − ak2 ≤ θkx − bk2 } for a 6= b. Show that S is a halfspace for θ = 1, convex for θ < 1. Give an example to prove that S can be non-convex for θ > 1. 17. Find the separating hyperplane between the two sets C = {x ∈ R2 |x2 ≤ 0} and D = {x ∈ R2+ |x1 x2 ≥ 1}. 18. Express the following norm balls as intersection half-spaces (a) {x ∈ Rn |kxk2 ≤ 1}, (b) {x ∈ Rn |kxk1 ≤ 1}, and (c) {x ∈ Rn |kxk∞ ≤ 1}. 19. Which of the following sets are convex (provide proof or counterexample) (a) {x ∈ R2 |x21 + 2ix1 x2 + i2 x22 ≤ 1 ∀ i = 1, 2, . . . , 10} (b) {x ∈ R2 |x21 + ix1 x2 + i2 x22 ≤ 1 ∀ i = 1, 2, . . . , 10} (c) {x ∈ Rn | mini xi = 1} (d) 20. Show that the perspective function transformation, P(C) := {x/t|[xT t]T ∈ C} preserves convexity. 21. Show that the set {x ∈ Rn |(Ax + b)T (Ax + b) ≤ (cT x + d)2 , cT x + d > 0} is a convex cone.

EE609

Assignment 3

January 16, 2015

Weight: 1% if submitted by Jan 22, 2014. Assignment will not be graded. No need to submit practice problems.

1 Assignment Problems 1. Starting from Jensen’s inequality, show that xθ y 1−θ ≤ θx + (1 − θ)y. 2. Consider a differentiable function f : R → R with R+ ⊆ dom f , and its running average, defined as Z 1 x f (t)dt (1) F (x) = x 0 with dom F = R++ . Show that F (x) is convex if f (x) is convex. P 3. Show that the harmonic mean f (x) = ( ni=1 1/xi )−1 is concave.

4. Define x[j] as the j-th largest component of x; e.g., x[1] = maxi {xi }, and x[n] =Pmini {xi }. Given any non-negative numbers α1 ≥ α2 ≥ . . . ≥ αr ≥ 0, prove that f (x) = ri=1 αi x[i] is convex.

n 5. Prove Pn the reverse Jensen’s inequality for a convex f with dom f = R , λi > 0 and λ1 − i=2 λi = 1

f (λ1 x1 − λ2 x2 − . . . − λn xn ) ≥ λ1 f (x1 ) − λ2 f (x2 ) − . . . − λn f (xn )

(2)

2 Practice Problems 6. Consider an increasing and convex function f : R → R with dom f = {a, b}. Let the function g : R → R denote its inverse, i.e., g(f (x)) = x for a ≤ x ≤ b and dom g = {f (a), f (b)}. Show that g is a concave function. 7. Show that the running average of the non-differentiable function f (x) is also convex. Since f is not differentiable, you must use the zeroth order condition to prove the convexity of F . 8. Give an example of a function f (x) whose epigraph is (a) half-space, (b) norm cone, and (c) polyhedron. 9. Give an example of a concave positive function (f (x) > 0) with domain Rn . 10. Show that the function f (x) is convex if and only if the function f (a + tb) is convex for all a + tb ∈ dom f and t. P 11. Show that the harmonic function: f (x) = ( ni=1 xai )1/a for a < 1, and a 6= 0 is concave. P 12. Show that the entropy function H(x) = − ni=1 xi log(xi ) with dom H = {x ∈ Rn++ |1T x = 1} is concave. 13. Determine if the following functions with dom f = R2++ are convex or concave (a) f (x1 , x2 ) = x1 x2 (b) f (x1 , x2 ) = x1 /x2 (c) f (x1 , x2 ) = x21 /x2

EE609

Assignment 3, Page 2 of 2

January 16, 2015

(d) f (x1 , x2 ) = xα1 x21−α for 0 ≤ α ≤ 1. √ (e) f (x1 , x2 = x1 x2 P 14. Show that the harmonic mean f (x) = ( ni=1 1/xi )−1 is concave.

15. Show that the function f (x, t) = xT x/t on dom f = Rn × R++ is convex in (x, t). 16. Show that the function f (x, s, t) = log(st − xT x) is convex on dom f = {(x, s, t) ∈ Rn+2 |st > xT x, s, t > 0}. 17. Show that the function f (x, t) = kxk33 /t2 is convex on {(x, t)|t > 0}. 18. Show that the function f (x) =

kAx+bk cT x+d

is convex on {x|cT x + d > 0}.

19. Prove the following for non-differentiable functions f and g. (a) If f and g are convex, both nondecreasing (or nonincreasing), and positive functions on an interval, then f g is convex. (b) If f , g are concave, positive, with one nondecreasing and the other nonincreasing, then f g is concave. (c) If f is convex, nondecreasing, and positive, and g is concave, nonincreasing, and positive, then f /g is convex. (d) If f is nonnegative and convex, and g is positive and concave, then f 2 /g is convex. 20. A function f (x) is log-concave if log(f (x)) is a concave function on {x|f (x) > 0}. Show that the Gaussian cdf F (x) is log-concave for all x. 21. Show that the maximum of a convex function f over the box B := {xl ≤ x ≤ xu } is achieved at one of its 2n vertices. 22. Using the following intermediate steps, show that the function f (x, t) = log(t2 − xT x) over dom f = {(x, t) ∈ Rn+1 |t > kxk2 } is convex. (a) Show that t − xT x/t is convex over dom f .

(b) Show that − log(t − xT x/t) is convex over dom f . 23. Show that the following function is convex on {x|kxk2 ≤ 1} f (x) =

kAx − bk22 1 − kxk22

(3)

24. Show that the weighted geometric mean f (x) =

n Y

xαk k

(4)

i=1

with dom f = Rn++ is concave for αk ≥ 0 and 25. Show that the Huber function f (x) = is convex on Rn .

(

Pn

i=1 αk

kxk22 /2 kxk22 − 1/2

= 1.

kxk2 ≤ 1 kxk2 > 1

(5)

EE609

Assignment 4

January 26, 2015

No submission required.

1 Assignment Problems 1. What is the solution of the following linear program min cT x

(1)

s. t. 0 ≤ xi ≤ 1

i = 1, . . . , n.

(2)

2. Consider the following linear program min cT x

(3)

s. t. Ax ≤ b

(4)

where A is square and full rank. (a) When is the problem infeasible? (b) When is the problem unbounded below? (c) When does the problem have a finite solution, and what is it? 3. Show that any linear programming problem can be expressed as min cT x

(5)

s. t. Ax = b

(6)

xi ≥ 0

i = 1, . . . , n

(7)

4. Solve the following quadratic optimization problem min cT x

(8)

s. t. kxk2 ≤ 1

(9)

5. Consider the sinusoidal measurement model in Gaussian noise described as, y(n) = αd + αc cos(2πf0 n) + αs sin(2πf0 n) + w(n), 0 ≤ n ≤ N − 1,

(10)

  where w(n) is additive white Gaussian noise and E |w(n)|2 = σn2 . Answer the questions that follow (a) Formulate the LS estimation problem for parameters αd , αc , αs . (b) Derive the LS estimator of αd , αc , αs with a suitable approximation for large N .

EE609

Assignment 4, Page 2 of 3

January 26, 2015

2 Practice Problems 6. Consider the following linear program min cT x

(11)

s. t. Ax = b

(12)

(a) When is the problem infeasible? (b) When is the problem unbounded below? (c) When does the problem have a finite solution, and what is it? 7. Consider the following linear program min cT x

(13)

T

s. t. a x ≤ b

(14)

where a 6= 0. (a) When is the problem infeasible? (b) When is the problem unbounded below? (c) When does the problem have a finite solution, and what is it? 8. Given that ci ≥ 0 for i = 1, 2, . . . , n, solve the following problem min cT x

(15)

s. t. ℓi ≤ xi ≤ bi

i = 1, . . . , n

(16)

where ℓi ≤ bi for all i = 1, 2, . . . , n. 9. What is the solution of the following linear program min cT x n X xi = 2 s. t.

(17) (18)

i=1

0 ≤ xi ≤ 1

i = 1, . . . , n

(19)

10. What is the solution of the following linear program min cT x n X xi = 1 s. t.

(20) (21)

i=1

xi ≥ 0

i = 1, . . . , n

(22)

11. Consider the two sets: S1 = {v1 , v2 , . . . , vp }

(23)

S2 = {u1 , u2 , . . . , uq }

(24)

EE609

Assignment 4, Page 3 of 3

January 26, 2015

Formulate the linear optimization problem to determine the separating hyperplane, i.e., find a ∈ Rn and b ∈ R such that aT v i ≤ b

i = 1, . . . , p

(25)

aT u i ≥ b

i = 1, . . . , q

(26)

Ensure that your problem excludes the trivial solution a = b = 0. 12. Consider the following quadratic optimization problem min cT x

(27)

T

s. t. x Ax ≤ 1

(28)

Let λi for i = 1, 2, . . . n be the real eigenvalues of a symmetric matrix A. What is the solution of this problem for the following cases. (a) If λi > 0 for i = 1, 2, . . . n. (b) If λ1 < 0, while λi > 0 for i = 2, . . . n. (c) If λ1 = 0, while λi > 0 for i = 2, . . . n. 13. Solve the following optimization problem for A ≻ 0, min cT x

(29) T

s. t. (x − xc ) A(x − xc ) ≤ 1

(30)

14. Solve the following optimization problem for A ≻ 0, min xT Ax s. t.

kxk22

≤1

(31) (32)

Hint: Given that the eigenvalue decomposition A = UΣUT , use the change of variable y = UT x.