Please copy and paste this embed script to where you want to embed

Problem Set 1 January 7, 2014 1.

(a) Prove that for rectangular matrices B, D, and invertible matrices A, C A(A + BCD)−1 = I − B(C−1 + DA−1 B)−1 DA−1 . (b) Using the above result, prove that (A−1 + BT C−1 B)−1 BT C−1 = ABT (BABT + C)−1 . (c) Find an expression for (A + xyT )−1 in terms of A−1 , x and y for invertible matrix A and (column) vectors x and y.

2. Let λi (A) denote an eigenvalue of A (a) λi (AB) = λi (BA) where A, B are not necessarily square. (b) If A = AT , find the following in terms of λi (A) i. ii. iii. iv.

Tr(Ap ). λi (I + cA). λi (A − cI). λi (A−1 ).

3. The notation A ≻ 0 denotes the fact that A is positive definite. Prove the following results for A ≻ 0: (a) A−1 ≻ 0

(b) [A]ii ≥ 0 for all i, where [A]ii denotes the i-th diagonal entry of A. (c) For any B, rank(BABT ) = rank(B).

(d) If B is full row-rank, then BABT ≻ 0.

(e) If A 0, then for any matrix X, XAXT = 0 ⇒ AX = 0.

4. Prove the following for the operator norm (with a = b, and all vector norms are k.ka ). (a) kIka,a = 1

(b) kAka,a = supkxk=1 kAxka (c) kAka,a = supkxk6=0

kAxk . kxk

(d) Let AT A 0, then kAk22,2 = maxi λi (AT A) (e) kAk1,1 = maxj

P

i |[A]i,j |

(f) kAxk ≤ kAka,a kxk

(g) kABka,a ≤ kAka,a kBka,a 5. Consider X = {x ∈ Rn |Ax = b} for A ∈ Rm×n and b ∈ Rm with b ∈ R(A). Prove that the following two statements are equivalent (i.e., (a) ⇒ (b) and (b) ⇒ (a)) (a) cT x = d for all x ∈ X

(b) There exists v ∈ Rm such that c = AT v and d = bT v. 6. Prove that (a) ku + vk22 = kuk22 + kvk22 if and only if uT v = 0.

(b) 2 < a, b > +2 < x, y >=< a + x, b + y > + < a − x, b − y > √ (c) kxk1 ≤ n kxk2

(d) kxk1 ≥ kxk2 ≥ kxk∞

7. Consider A ∈ Rm×n where m ≥ n and rank of A is n. Suppose there exists B such that BA = I and that AB = BT AT . (a) Let X = AB. Show that i. X2 = X ii. Tr(X) = rank(X) (b) Find an expression for B in terms of A. (c) For any two vectors b ∈ Rn and c ∈ Rm , prove that kAb − ck2 ≥ kABc − ck2 .

EE609

Assignment 2

January 8, 2015

Weight: 1% if submitted by Jan 15, 2014. Assignment will not be graded. No need to submit practice problems.

1 Assignment Problems 1. Show that the set {x ∈ Rn |kx − x1 k2 ≤ kx − x2 k2 } is actually a half space. and express it in canonical form. 2. Show that the set {x ∈ Rn |kxk1 ≤ 2} is a polyhedra and express it as an intersection of half spaces (and hyperplanes, if required). 3. Show that the set {x ∈ R2++ |x1 x2 ≥ 1} is convex and express it as an intersection of infinite number of half-spaces. 4. Is the following set convex: {x ∈ Rn |x ≥ 0, xT y ≤ 1 for all y with kyk2 = 1}? Show that the set {x ∈ Rn |xT y ≤ 1 for all y with y ∈ C} is always convex, even if C is not convex. 5. Consider the two-dimensional positive semidefinite cone S2+ defined as {X = [ xy yz ] |x, y, z ∈ R, X 0}

(1)

Show that it can equivalently be expressed as {x, y, z ∈ R|x ≥ 0, z ≥ 0, xz ≥ y 2 }.

2 Practice Problems 6. Show that a set is affine if and only if its intersection with any line is affine. 7. What is the minimum distance between two parallel halfspaces {x ∈ Rn |aT x ≤ b1 } and {x ∈ Rn |aT x ≥ b2 } (Hint: it depends on where the origin is)? ˜, b, and ˜b, is the following true? 8. Under what conditions on a, a {x ∈ Rn |aT x ≤ b} ⊆ {x ∈ Rn |˜ aT x ≤ ˜b}

(2)

9. For m < n, let b ∈ Rm and A ∈ Rm×n be full row rank. Show that any affine set {x ∈ Rn |Ax = b} can be expressed in the form {Cu + v|u ∈ Rm }. For example, the set {x ∈ R2 |x1 + x2 = 1} can be expressed as {[u 1 − u]T |u ∈ R} 10. What is the affine hull of the set {x ∈ R3 |x21 + x22 = 1, x3 = 1}. 11. Is the following set affine: {x ∈ Rn |kx − x1 k1 ≤ kx − x2 k1 }? 12. Given two vectors y, z ∈ Rn , consider the set S{a,b} = {x ∈ Rn |x = ay + bz}. Show that the following set is a polyhedra, and find its boundaries. [ [ S{a,b} (3) a∈[−1,1] b∈[−1,1]

13. Show that the set {x ∈ Rn |kxk∞ ≤ 1} is a polyhedra and express it as an intersection of half spaces (and hyperplanes, if required).

EE609

Assignment 2, Page 2 of 2

January 8, 2015

14. Show that the set of all doubly stochastic matrices is convex polyhedral in Rn×n . A doubly stochastic matrix is a square matrix with nonnegative entries with the property that the sum of entries in every row and column is exactly 1. 15. What kind of set is C = {x ∈ R2 |p(0) = 1, |p(t)| ≤ 1, for 1 ≤ t ≤ 2} where p(t) = x1 + x2 t? 16. Given θ, consider the set S = {x ∈ Rn |kx − ak2 ≤ θkx − bk2 } for a 6= b. Show that S is a halfspace for θ = 1, convex for θ < 1. Give an example to prove that S can be non-convex for θ > 1. 17. Find the separating hyperplane between the two sets C = {x ∈ R2 |x2 ≤ 0} and D = {x ∈ R2+ |x1 x2 ≥ 1}. 18. Express the following norm balls as intersection half-spaces (a) {x ∈ Rn |kxk2 ≤ 1}, (b) {x ∈ Rn |kxk1 ≤ 1}, and (c) {x ∈ Rn |kxk∞ ≤ 1}. 19. Which of the following sets are convex (provide proof or counterexample) (a) {x ∈ R2 |x21 + 2ix1 x2 + i2 x22 ≤ 1 ∀ i = 1, 2, . . . , 10} (b) {x ∈ R2 |x21 + ix1 x2 + i2 x22 ≤ 1 ∀ i = 1, 2, . . . , 10} (c) {x ∈ Rn | mini xi = 1} (d) 20. Show that the perspective function transformation, P(C) := {x/t|[xT t]T ∈ C} preserves convexity. 21. Show that the set {x ∈ Rn |(Ax + b)T (Ax + b) ≤ (cT x + d)2 , cT x + d > 0} is a convex cone.

EE609

Assignment 3

January 16, 2015

Weight: 1% if submitted by Jan 22, 2014. Assignment will not be graded. No need to submit practice problems.

1 Assignment Problems 1. Starting from Jensen’s inequality, show that xθ y 1−θ ≤ θx + (1 − θ)y. 2. Consider a differentiable function f : R → R with R+ ⊆ dom f , and its running average, defined as Z 1 x f (t)dt (1) F (x) = x 0 with dom F = R++ . Show that F (x) is convex if f (x) is convex. P 3. Show that the harmonic mean f (x) = ( ni=1 1/xi )−1 is concave.

4. Define x[j] as the j-th largest component of x; e.g., x[1] = maxi {xi }, and x[n] =Pmini {xi }. Given any non-negative numbers α1 ≥ α2 ≥ . . . ≥ αr ≥ 0, prove that f (x) = ri=1 αi x[i] is convex.

n 5. Prove Pn the reverse Jensen’s inequality for a convex f with dom f = R , λi > 0 and λ1 − i=2 λi = 1

f (λ1 x1 − λ2 x2 − . . . − λn xn ) ≥ λ1 f (x1 ) − λ2 f (x2 ) − . . . − λn f (xn )

(2)

2 Practice Problems 6. Consider an increasing and convex function f : R → R with dom f = {a, b}. Let the function g : R → R denote its inverse, i.e., g(f (x)) = x for a ≤ x ≤ b and dom g = {f (a), f (b)}. Show that g is a concave function. 7. Show that the running average of the non-differentiable function f (x) is also convex. Since f is not differentiable, you must use the zeroth order condition to prove the convexity of F . 8. Give an example of a function f (x) whose epigraph is (a) half-space, (b) norm cone, and (c) polyhedron. 9. Give an example of a concave positive function (f (x) > 0) with domain Rn . 10. Show that the function f (x) is convex if and only if the function f (a + tb) is convex for all a + tb ∈ dom f and t. P 11. Show that the harmonic function: f (x) = ( ni=1 xai )1/a for a < 1, and a 6= 0 is concave. P 12. Show that the entropy function H(x) = − ni=1 xi log(xi ) with dom H = {x ∈ Rn++ |1T x = 1} is concave. 13. Determine if the following functions with dom f = R2++ are convex or concave (a) f (x1 , x2 ) = x1 x2 (b) f (x1 , x2 ) = x1 /x2 (c) f (x1 , x2 ) = x21 /x2

EE609

Assignment 3, Page 2 of 2

January 16, 2015

(d) f (x1 , x2 ) = xα1 x21−α for 0 ≤ α ≤ 1. √ (e) f (x1 , x2 = x1 x2 P 14. Show that the harmonic mean f (x) = ( ni=1 1/xi )−1 is concave.

15. Show that the function f (x, t) = xT x/t on dom f = Rn × R++ is convex in (x, t). 16. Show that the function f (x, s, t) = log(st − xT x) is convex on dom f = {(x, s, t) ∈ Rn+2 |st > xT x, s, t > 0}. 17. Show that the function f (x, t) = kxk33 /t2 is convex on {(x, t)|t > 0}. 18. Show that the function f (x) =

kAx+bk cT x+d

is convex on {x|cT x + d > 0}.

19. Prove the following for non-differentiable functions f and g. (a) If f and g are convex, both nondecreasing (or nonincreasing), and positive functions on an interval, then f g is convex. (b) If f , g are concave, positive, with one nondecreasing and the other nonincreasing, then f g is concave. (c) If f is convex, nondecreasing, and positive, and g is concave, nonincreasing, and positive, then f /g is convex. (d) If f is nonnegative and convex, and g is positive and concave, then f 2 /g is convex. 20. A function f (x) is log-concave if log(f (x)) is a concave function on {x|f (x) > 0}. Show that the Gaussian cdf F (x) is log-concave for all x. 21. Show that the maximum of a convex function f over the box B := {xl ≤ x ≤ xu } is achieved at one of its 2n vertices. 22. Using the following intermediate steps, show that the function f (x, t) = log(t2 − xT x) over dom f = {(x, t) ∈ Rn+1 |t > kxk2 } is convex. (a) Show that t − xT x/t is convex over dom f .

(b) Show that − log(t − xT x/t) is convex over dom f . 23. Show that the following function is convex on {x|kxk2 ≤ 1} f (x) =

kAx − bk22 1 − kxk22

(3)

24. Show that the weighted geometric mean f (x) =

n Y

xαk k

(4)

i=1

with dom f = Rn++ is concave for αk ≥ 0 and 25. Show that the Huber function f (x) = is convex on Rn .

(

Pn

i=1 αk

kxk22 /2 kxk22 − 1/2

= 1.

kxk2 ≤ 1 kxk2 > 1

(5)

EE609

Assignment 4

January 26, 2015

No submission required.

1 Assignment Problems 1. What is the solution of the following linear program min cT x

(1)

s. t. 0 ≤ xi ≤ 1

i = 1, . . . , n.

(2)

2. Consider the following linear program min cT x

(3)

s. t. Ax ≤ b

(4)

where A is square and full rank. (a) When is the problem infeasible? (b) When is the problem unbounded below? (c) When does the problem have a finite solution, and what is it? 3. Show that any linear programming problem can be expressed as min cT x

(5)

s. t. Ax = b

(6)

xi ≥ 0

i = 1, . . . , n

(7)

4. Solve the following quadratic optimization problem min cT x

(8)

s. t. kxk2 ≤ 1

(9)

5. Consider the sinusoidal measurement model in Gaussian noise described as, y(n) = αd + αc cos(2πf0 n) + αs sin(2πf0 n) + w(n), 0 ≤ n ≤ N − 1,

(10)

where w(n) is additive white Gaussian noise and E |w(n)|2 = σn2 . Answer the questions that follow (a) Formulate the LS estimation problem for parameters αd , αc , αs . (b) Derive the LS estimator of αd , αc , αs with a suitable approximation for large N .

EE609

Assignment 4, Page 2 of 3

January 26, 2015

2 Practice Problems 6. Consider the following linear program min cT x

(11)

s. t. Ax = b

(12)

(a) When is the problem infeasible? (b) When is the problem unbounded below? (c) When does the problem have a finite solution, and what is it? 7. Consider the following linear program min cT x

(13)

T

s. t. a x ≤ b

(14)

where a 6= 0. (a) When is the problem infeasible? (b) When is the problem unbounded below? (c) When does the problem have a finite solution, and what is it? 8. Given that ci ≥ 0 for i = 1, 2, . . . , n, solve the following problem min cT x

(15)

s. t. ℓi ≤ xi ≤ bi

i = 1, . . . , n

(16)

where ℓi ≤ bi for all i = 1, 2, . . . , n. 9. What is the solution of the following linear program min cT x n X xi = 2 s. t.

(17) (18)

i=1

0 ≤ xi ≤ 1

i = 1, . . . , n

(19)

10. What is the solution of the following linear program min cT x n X xi = 1 s. t.

(20) (21)

i=1

xi ≥ 0

i = 1, . . . , n

(22)

11. Consider the two sets: S1 = {v1 , v2 , . . . , vp }

(23)

S2 = {u1 , u2 , . . . , uq }

(24)

EE609

Assignment 4, Page 3 of 3

January 26, 2015

Formulate the linear optimization problem to determine the separating hyperplane, i.e., find a ∈ Rn and b ∈ R such that aT v i ≤ b

i = 1, . . . , p

(25)

aT u i ≥ b

i = 1, . . . , q

(26)

Ensure that your problem excludes the trivial solution a = b = 0. 12. Consider the following quadratic optimization problem min cT x

(27)

T

s. t. x Ax ≤ 1

(28)

Let λi for i = 1, 2, . . . n be the real eigenvalues of a symmetric matrix A. What is the solution of this problem for the following cases. (a) If λi > 0 for i = 1, 2, . . . n. (b) If λ1 < 0, while λi > 0 for i = 2, . . . n. (c) If λ1 = 0, while λi > 0 for i = 2, . . . n. 13. Solve the following optimization problem for A ≻ 0, min cT x

(29) T

s. t. (x − xc ) A(x − xc ) ≤ 1

(30)

14. Solve the following optimization problem for A ≻ 0, min xT Ax s. t.

kxk22

≤1

(31) (32)

Hint: Given that the eigenvalue decomposition A = UΣUT , use the change of variable y = UT x.

View more...
(a) Prove that for rectangular matrices B, D, and invertible matrices A, C A(A + BCD)−1 = I − B(C−1 + DA−1 B)−1 DA−1 . (b) Using the above result, prove that (A−1 + BT C−1 B)−1 BT C−1 = ABT (BABT + C)−1 . (c) Find an expression for (A + xyT )−1 in terms of A−1 , x and y for invertible matrix A and (column) vectors x and y.

2. Let λi (A) denote an eigenvalue of A (a) λi (AB) = λi (BA) where A, B are not necessarily square. (b) If A = AT , find the following in terms of λi (A) i. ii. iii. iv.

Tr(Ap ). λi (I + cA). λi (A − cI). λi (A−1 ).

3. The notation A ≻ 0 denotes the fact that A is positive definite. Prove the following results for A ≻ 0: (a) A−1 ≻ 0

(b) [A]ii ≥ 0 for all i, where [A]ii denotes the i-th diagonal entry of A. (c) For any B, rank(BABT ) = rank(B).

(d) If B is full row-rank, then BABT ≻ 0.

(e) If A 0, then for any matrix X, XAXT = 0 ⇒ AX = 0.

4. Prove the following for the operator norm (with a = b, and all vector norms are k.ka ). (a) kIka,a = 1

(b) kAka,a = supkxk=1 kAxka (c) kAka,a = supkxk6=0

kAxk . kxk

(d) Let AT A 0, then kAk22,2 = maxi λi (AT A) (e) kAk1,1 = maxj

P

i |[A]i,j |

(f) kAxk ≤ kAka,a kxk

(g) kABka,a ≤ kAka,a kBka,a 5. Consider X = {x ∈ Rn |Ax = b} for A ∈ Rm×n and b ∈ Rm with b ∈ R(A). Prove that the following two statements are equivalent (i.e., (a) ⇒ (b) and (b) ⇒ (a)) (a) cT x = d for all x ∈ X

(b) There exists v ∈ Rm such that c = AT v and d = bT v. 6. Prove that (a) ku + vk22 = kuk22 + kvk22 if and only if uT v = 0.

(b) 2 < a, b > +2 < x, y >=< a + x, b + y > + < a − x, b − y > √ (c) kxk1 ≤ n kxk2

(d) kxk1 ≥ kxk2 ≥ kxk∞

7. Consider A ∈ Rm×n where m ≥ n and rank of A is n. Suppose there exists B such that BA = I and that AB = BT AT . (a) Let X = AB. Show that i. X2 = X ii. Tr(X) = rank(X) (b) Find an expression for B in terms of A. (c) For any two vectors b ∈ Rn and c ∈ Rm , prove that kAb − ck2 ≥ kABc − ck2 .

EE609

Assignment 2

January 8, 2015

Weight: 1% if submitted by Jan 15, 2014. Assignment will not be graded. No need to submit practice problems.

1 Assignment Problems 1. Show that the set {x ∈ Rn |kx − x1 k2 ≤ kx − x2 k2 } is actually a half space. and express it in canonical form. 2. Show that the set {x ∈ Rn |kxk1 ≤ 2} is a polyhedra and express it as an intersection of half spaces (and hyperplanes, if required). 3. Show that the set {x ∈ R2++ |x1 x2 ≥ 1} is convex and express it as an intersection of infinite number of half-spaces. 4. Is the following set convex: {x ∈ Rn |x ≥ 0, xT y ≤ 1 for all y with kyk2 = 1}? Show that the set {x ∈ Rn |xT y ≤ 1 for all y with y ∈ C} is always convex, even if C is not convex. 5. Consider the two-dimensional positive semidefinite cone S2+ defined as {X = [ xy yz ] |x, y, z ∈ R, X 0}

(1)

Show that it can equivalently be expressed as {x, y, z ∈ R|x ≥ 0, z ≥ 0, xz ≥ y 2 }.

2 Practice Problems 6. Show that a set is affine if and only if its intersection with any line is affine. 7. What is the minimum distance between two parallel halfspaces {x ∈ Rn |aT x ≤ b1 } and {x ∈ Rn |aT x ≥ b2 } (Hint: it depends on where the origin is)? ˜, b, and ˜b, is the following true? 8. Under what conditions on a, a {x ∈ Rn |aT x ≤ b} ⊆ {x ∈ Rn |˜ aT x ≤ ˜b}

(2)

9. For m < n, let b ∈ Rm and A ∈ Rm×n be full row rank. Show that any affine set {x ∈ Rn |Ax = b} can be expressed in the form {Cu + v|u ∈ Rm }. For example, the set {x ∈ R2 |x1 + x2 = 1} can be expressed as {[u 1 − u]T |u ∈ R} 10. What is the affine hull of the set {x ∈ R3 |x21 + x22 = 1, x3 = 1}. 11. Is the following set affine: {x ∈ Rn |kx − x1 k1 ≤ kx − x2 k1 }? 12. Given two vectors y, z ∈ Rn , consider the set S{a,b} = {x ∈ Rn |x = ay + bz}. Show that the following set is a polyhedra, and find its boundaries. [ [ S{a,b} (3) a∈[−1,1] b∈[−1,1]

13. Show that the set {x ∈ Rn |kxk∞ ≤ 1} is a polyhedra and express it as an intersection of half spaces (and hyperplanes, if required).

EE609

Assignment 2, Page 2 of 2

January 8, 2015

14. Show that the set of all doubly stochastic matrices is convex polyhedral in Rn×n . A doubly stochastic matrix is a square matrix with nonnegative entries with the property that the sum of entries in every row and column is exactly 1. 15. What kind of set is C = {x ∈ R2 |p(0) = 1, |p(t)| ≤ 1, for 1 ≤ t ≤ 2} where p(t) = x1 + x2 t? 16. Given θ, consider the set S = {x ∈ Rn |kx − ak2 ≤ θkx − bk2 } for a 6= b. Show that S is a halfspace for θ = 1, convex for θ < 1. Give an example to prove that S can be non-convex for θ > 1. 17. Find the separating hyperplane between the two sets C = {x ∈ R2 |x2 ≤ 0} and D = {x ∈ R2+ |x1 x2 ≥ 1}. 18. Express the following norm balls as intersection half-spaces (a) {x ∈ Rn |kxk2 ≤ 1}, (b) {x ∈ Rn |kxk1 ≤ 1}, and (c) {x ∈ Rn |kxk∞ ≤ 1}. 19. Which of the following sets are convex (provide proof or counterexample) (a) {x ∈ R2 |x21 + 2ix1 x2 + i2 x22 ≤ 1 ∀ i = 1, 2, . . . , 10} (b) {x ∈ R2 |x21 + ix1 x2 + i2 x22 ≤ 1 ∀ i = 1, 2, . . . , 10} (c) {x ∈ Rn | mini xi = 1} (d) 20. Show that the perspective function transformation, P(C) := {x/t|[xT t]T ∈ C} preserves convexity. 21. Show that the set {x ∈ Rn |(Ax + b)T (Ax + b) ≤ (cT x + d)2 , cT x + d > 0} is a convex cone.

EE609

Assignment 3

January 16, 2015

Weight: 1% if submitted by Jan 22, 2014. Assignment will not be graded. No need to submit practice problems.

1 Assignment Problems 1. Starting from Jensen’s inequality, show that xθ y 1−θ ≤ θx + (1 − θ)y. 2. Consider a differentiable function f : R → R with R+ ⊆ dom f , and its running average, defined as Z 1 x f (t)dt (1) F (x) = x 0 with dom F = R++ . Show that F (x) is convex if f (x) is convex. P 3. Show that the harmonic mean f (x) = ( ni=1 1/xi )−1 is concave.

4. Define x[j] as the j-th largest component of x; e.g., x[1] = maxi {xi }, and x[n] =Pmini {xi }. Given any non-negative numbers α1 ≥ α2 ≥ . . . ≥ αr ≥ 0, prove that f (x) = ri=1 αi x[i] is convex.

n 5. Prove Pn the reverse Jensen’s inequality for a convex f with dom f = R , λi > 0 and λ1 − i=2 λi = 1

f (λ1 x1 − λ2 x2 − . . . − λn xn ) ≥ λ1 f (x1 ) − λ2 f (x2 ) − . . . − λn f (xn )

(2)

2 Practice Problems 6. Consider an increasing and convex function f : R → R with dom f = {a, b}. Let the function g : R → R denote its inverse, i.e., g(f (x)) = x for a ≤ x ≤ b and dom g = {f (a), f (b)}. Show that g is a concave function. 7. Show that the running average of the non-differentiable function f (x) is also convex. Since f is not differentiable, you must use the zeroth order condition to prove the convexity of F . 8. Give an example of a function f (x) whose epigraph is (a) half-space, (b) norm cone, and (c) polyhedron. 9. Give an example of a concave positive function (f (x) > 0) with domain Rn . 10. Show that the function f (x) is convex if and only if the function f (a + tb) is convex for all a + tb ∈ dom f and t. P 11. Show that the harmonic function: f (x) = ( ni=1 xai )1/a for a < 1, and a 6= 0 is concave. P 12. Show that the entropy function H(x) = − ni=1 xi log(xi ) with dom H = {x ∈ Rn++ |1T x = 1} is concave. 13. Determine if the following functions with dom f = R2++ are convex or concave (a) f (x1 , x2 ) = x1 x2 (b) f (x1 , x2 ) = x1 /x2 (c) f (x1 , x2 ) = x21 /x2

EE609

Assignment 3, Page 2 of 2

January 16, 2015

(d) f (x1 , x2 ) = xα1 x21−α for 0 ≤ α ≤ 1. √ (e) f (x1 , x2 = x1 x2 P 14. Show that the harmonic mean f (x) = ( ni=1 1/xi )−1 is concave.

15. Show that the function f (x, t) = xT x/t on dom f = Rn × R++ is convex in (x, t). 16. Show that the function f (x, s, t) = log(st − xT x) is convex on dom f = {(x, s, t) ∈ Rn+2 |st > xT x, s, t > 0}. 17. Show that the function f (x, t) = kxk33 /t2 is convex on {(x, t)|t > 0}. 18. Show that the function f (x) =

kAx+bk cT x+d

is convex on {x|cT x + d > 0}.

19. Prove the following for non-differentiable functions f and g. (a) If f and g are convex, both nondecreasing (or nonincreasing), and positive functions on an interval, then f g is convex. (b) If f , g are concave, positive, with one nondecreasing and the other nonincreasing, then f g is concave. (c) If f is convex, nondecreasing, and positive, and g is concave, nonincreasing, and positive, then f /g is convex. (d) If f is nonnegative and convex, and g is positive and concave, then f 2 /g is convex. 20. A function f (x) is log-concave if log(f (x)) is a concave function on {x|f (x) > 0}. Show that the Gaussian cdf F (x) is log-concave for all x. 21. Show that the maximum of a convex function f over the box B := {xl ≤ x ≤ xu } is achieved at one of its 2n vertices. 22. Using the following intermediate steps, show that the function f (x, t) = log(t2 − xT x) over dom f = {(x, t) ∈ Rn+1 |t > kxk2 } is convex. (a) Show that t − xT x/t is convex over dom f .

(b) Show that − log(t − xT x/t) is convex over dom f . 23. Show that the following function is convex on {x|kxk2 ≤ 1} f (x) =

kAx − bk22 1 − kxk22

(3)

24. Show that the weighted geometric mean f (x) =

n Y

xαk k

(4)

i=1

with dom f = Rn++ is concave for αk ≥ 0 and 25. Show that the Huber function f (x) = is convex on Rn .

(

Pn

i=1 αk

kxk22 /2 kxk22 − 1/2

= 1.

kxk2 ≤ 1 kxk2 > 1

(5)

EE609

Assignment 4

January 26, 2015

No submission required.

1 Assignment Problems 1. What is the solution of the following linear program min cT x

(1)

s. t. 0 ≤ xi ≤ 1

i = 1, . . . , n.

(2)

2. Consider the following linear program min cT x

(3)

s. t. Ax ≤ b

(4)

where A is square and full rank. (a) When is the problem infeasible? (b) When is the problem unbounded below? (c) When does the problem have a finite solution, and what is it? 3. Show that any linear programming problem can be expressed as min cT x

(5)

s. t. Ax = b

(6)

xi ≥ 0

i = 1, . . . , n

(7)

4. Solve the following quadratic optimization problem min cT x

(8)

s. t. kxk2 ≤ 1

(9)

5. Consider the sinusoidal measurement model in Gaussian noise described as, y(n) = αd + αc cos(2πf0 n) + αs sin(2πf0 n) + w(n), 0 ≤ n ≤ N − 1,

(10)

where w(n) is additive white Gaussian noise and E |w(n)|2 = σn2 . Answer the questions that follow (a) Formulate the LS estimation problem for parameters αd , αc , αs . (b) Derive the LS estimator of αd , αc , αs with a suitable approximation for large N .

EE609

Assignment 4, Page 2 of 3

January 26, 2015

2 Practice Problems 6. Consider the following linear program min cT x

(11)

s. t. Ax = b

(12)

(a) When is the problem infeasible? (b) When is the problem unbounded below? (c) When does the problem have a finite solution, and what is it? 7. Consider the following linear program min cT x

(13)

T

s. t. a x ≤ b

(14)

where a 6= 0. (a) When is the problem infeasible? (b) When is the problem unbounded below? (c) When does the problem have a finite solution, and what is it? 8. Given that ci ≥ 0 for i = 1, 2, . . . , n, solve the following problem min cT x

(15)

s. t. ℓi ≤ xi ≤ bi

i = 1, . . . , n

(16)

where ℓi ≤ bi for all i = 1, 2, . . . , n. 9. What is the solution of the following linear program min cT x n X xi = 2 s. t.

(17) (18)

i=1

0 ≤ xi ≤ 1

i = 1, . . . , n

(19)

10. What is the solution of the following linear program min cT x n X xi = 1 s. t.

(20) (21)

i=1

xi ≥ 0

i = 1, . . . , n

(22)

11. Consider the two sets: S1 = {v1 , v2 , . . . , vp }

(23)

S2 = {u1 , u2 , . . . , uq }

(24)

EE609

Assignment 4, Page 3 of 3

January 26, 2015

Formulate the linear optimization problem to determine the separating hyperplane, i.e., find a ∈ Rn and b ∈ R such that aT v i ≤ b

i = 1, . . . , p

(25)

aT u i ≥ b

i = 1, . . . , q

(26)

Ensure that your problem excludes the trivial solution a = b = 0. 12. Consider the following quadratic optimization problem min cT x

(27)

T

s. t. x Ax ≤ 1

(28)

Let λi for i = 1, 2, . . . n be the real eigenvalues of a symmetric matrix A. What is the solution of this problem for the following cases. (a) If λi > 0 for i = 1, 2, . . . n. (b) If λ1 < 0, while λi > 0 for i = 2, . . . n. (c) If λ1 = 0, while λi > 0 for i = 2, . . . n. 13. Solve the following optimization problem for A ≻ 0, min cT x

(29) T

s. t. (x − xc ) A(x − xc ) ≤ 1

(30)

14. Solve the following optimization problem for A ≻ 0, min xT Ax s. t.

kxk22

≤1

(31) (32)

Hint: Given that the eigenvalue decomposition A = UΣUT , use the change of variable y = UT x.

Thank you for interesting in our services. We are a non-profit group that run this website to share documents. We need your help to maintenance this website.

To keep our site running, we need your help to cover our server cost (about $400/m), a small donation will help us a lot.