Top Twelve Ways to Speed Up Your Liferay Deployment

TOP TWELVE WAYS TO SPEED UP YOUR LIFERAY DEPLOYMENT Copyright © 2000 ­ 2009 Liferay, Inc. All Rights Reserved. No material may be reproduced electronically or in print, duplicated, copied, sold, resold, or otherwise exploited for any commercial purpose without express written consent of Liferay, Inc.

Rich Sezov Knowledge Manager Liferay, Inc.

Liferay Portal: Open Source and Tweakable As an infrastructure portal, Liferay Portal can support over 3300 concurrent users on a single server with mean login times under ½ a second and maximum throughput of 79+ logins per second. In collaboration and social networking scenarios, each physical server supports over 1300 concurrent users at average transaction times of under 800ms. Liferay Portal’s WCM scales to beyond 150,000 concurrent users on a single Liferay Portal server with average transaction times under 50ms and 35% CPU utilization. Given sufficient database resources and efficient load balancing, Liferay Portal can scale linearly as one adds additional servers to a cluster.

How do I get Liferay to do that?!?

Top Twelve Ways to Speed Up Your Liferay Deployment

Number 12: Flood the data center with gamma radiation and then make the server angry by rebooting it over and over again.

Number 12: Adjust the server's thread pool and JDBC connection pool. Unfortunately, there's no magic number for this: it must be tuned based on usage. By default, Liferay is configured for a maximum of 100 database connections. For Tomcat, a good number is between 200 and 400 threads in the thread pool. YMMV: use a profiler and tune to the right number.

Number 11: Detach Jon Stewart's mouth from the network router.

Number 11: Turn off unused servlet filters. Servlet filters were introduced in the servlet specification 2.3. They dynamically intercept requests and transform them in some way. Liferay contains 17 servlet filters. Chances are, you don't need them all, so turn off the ones you aren't using!

Servlet Filters to Turn Off SSO CAS Filter: Are you using CAS for Single Sign-On? If not, you don't need this filter running. SSO NTLM Filter: Are your users authenticating via NTLM (NT LAN Manager)? If not, you don't need this filter running. SSO OpenSSO Filter: Are you using OpenSSO for Single Sign-On? If not, you don't need this filter running. Virtual Host Filter: Are you mapping domain names to communities or organizations? If not, turn this filter off. Sharepoint Filter: Are you using Liferay's Sharepoint functionality for saving documents directly to the portal? If not, this filter is not for you. Turn it off.

How do you turn off a servlet filter? Easy! Comment it out of the web.xml file: SSO Ntlm Filter com.liferay.portal.servlet.filters.sso.ntlm.NtlmFilter --> … SSO Ntlm Filter /c/portal/login -->

Number 10: Use EMM386 to put all the TSRs into high memory to free up—oh, never mind, wrong decade.

Number 10: Tune your JVM parameters. Again, there is nothing set in stone for this: you will have to go through the cycle of tune and profile, tune and profile until you get the parameters right. Java memory looks something like this: Young Generation Eden

From

To

Java Heap

Old Generation

Permanent Generation

Garbage Collection When Garbage Collection occurs, here's what happens: Young Generation X

X

X

X

X

To

Old Generation

Permanent Generation

When all of this is done, the space is compacted, so the memory is contiguous.

Serial vs. Parallel Garbage Collection By default, the JDK uses a serial garbage collector. When it runs, the garbage collector stops all application execution in order to do its job. This works really well for desktop-based, client applications which are running on one processor. For server-based, multi processor systems, you will perhaps want to switch to the parallel garbage collector known as the Concurrent Mark-Sweep collector (CMS). This collector makes one short pause in application execution to mark objects directly reachable from the application code. Then it allows the application to run while it marks all objects which are reachable from the set it marked. Finally, it adds another phase called the remark phase which finalizes marking by revisiting any objects modified while the application was running. It then sweeps through and garbage collects.

JVM Options NewSize, MaxNewSize: The initial size and the maximum size of the New or Young Generation. +UseParNewGC: Causes garbage collection to happen in parallel, using multiple CPUs. This decreases garbage collection overhead and increases application throughput. +UseConcMarkSweepGC: Use the Concurrent Mark-Sweep Garbage Collector. This uses shorter garbage collection pauses, and is good for applications that have a relatively large set of long-lived data, and that run on machines with two or more processors, such as web servers. +CMSParallelRemarkEnabled: For the CMS GC, enables the garbage collector to use multiple threads during the CMS remark phase. This decreases the pauses during this phase. ServivorRatio: Controls the size of the two survivor spaces. It's a ratio between the survivor space size and Eden. The default is 25. There's not much bang for the buck here, but it may need to be adjusted. ParallelGCThreads: The number of threads to use for parallel garbage collection. Should be equal to the number of CPU cores in your server.

Example Java Options String JAVA_OPTS="$JAVA_OPTS -XX:NewSize=700m -XX:MaxNewSize=700m -Xms2048m -Xmx2048m -XX:MaxPermSize=128m -XX:+UseParNewGC -XX: +UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=20 -XX:ParallelGCThreads=8"

Number 9: Tell the OS it's been featured on America's Most Wanted and—hey look, you better run: I see some FBI agents coming this way!

Number 9: Tune ehcache. Liferay uses ehcache, which is a cluster-aware, tunable cache. Caching greatly speeds up performance by reducing the number of times the application has to go grab something from the database. Liferay's cache comes tuned to default settings, but you may want to modify it to suit your web site. If you have a heavily trafficked message board, you may want to consider adjusting the cache for the message board.

Caching the Message Board

Tuning the Cache MaxElementsInMemory: Monitor the cache using a JMX Console, as you cannot guess at the right amount here. You can adjust the setting if you find the cache is full. TimeToIdleSeconds: This sets the time to idle for an element before it expires from the cache. Eternal: If eternal, timeouts are ignored and the element is never expired.

Other Cache Settings There are many, many other settings which can be used to tune the cache. You can, as an example, change the cache algorithm if it seems to be caching the wrong things. If we were to go over them all, we'd never get through to the rest of the top twelve; speaking of which....

Number 8: Take a queue from Speed: configure the server so that any time the pages per second drops below your defined threshold, it'll blow up. That'll get it moving.

Number 8: Lucene Index Writer Interval Whenever Liferay calls Lucene to index some content, it may create any number of files to do so. Depending on the content, these files can be large files or lots of small files. Every now and then, Liferay optimizes the index for reading by combining smaller files into larger files. You can change this behavior based on your use case. The property is lucene.optimize.interval If you are doing a lot of publishing and loading of data, make the number very high, like 1000. If you are doing mostly reads, make it low, like the default value of 100. Of course, the best thing is to move search out to a separate environment, such as Solr.

Number 7:

Reconfigure the dilithium chamber to accept a new crystal matrix.

Number 7: Replace Lucene altogether with Solr Apache says: –

Solr is an open source enterprise search server based on the Lucene Java search library, with XML/HTTP and JSON APIs, hit highlighting, faceted search, caching, replication, a web administration interface and many more features. It runs in a Java servlet container such as Tomcat.

Solr allows you to abstract out of your Liferay installation everything that has to do with search, and run search from a completely separate environment.

Installing Solr in 7 Steps Step 1: Install the Solr web application on a separate environment from your Liferay environment. Step 2: Grab the Solr plugin from Liferay and extract it to your file system. Step 3: Edit the file docroot/WEB-INF/src/META-INF/solr-spring.xml. Step 4: Change the URL in the following Spring bean configuration to point to your newly installed Solr box and the save the file:

Step 5: Copy the conf/schema.xml file to the $SOLR_HOME/conf folder on your newly installed Solr box. Step 6: Zip the plugin back up into a .war file. Start your Solr server. Deploy the plugin to Liferay. Step 7: Reindex your content.

Number 6: That's it, we have no other choice: go to Ludicrous Speed!

Number 6: Optimize Counter Increment One of the ways Liferay is able to support so many databases is that it does not use any single database's method of determining sequences for primary keys. Instead, Liferay includes its own counter utility which can be optimized. The default value: counter.increment=100 will cause Liferay to go to the database to update the counter only once for every 100 primary keys it needs to create. Each time the counter increments itself, it keeps track of the current set of available keys in an in-memory, cluster aware object. You could set this to a higher number to reduce the number of database calls for primary keys within Liferay.

Number 5: Use a Redundant Array of Impelling Rodents (RAIR).

Number 5: Use a Content Delivery Network A Content Delivery Network serves up static content from a location that is geographically close to the end user. This goes one step better than simply using a web server to serve up your static content, and is very simple to set up. cdn.host=[your CDN host here] The value should include the full protocol and port number if the CDN does not use the standard HTTP and HTTPS ports. Liferay.com is configured this way.

Number 4:

Hire a good smuggler to smuggle your goods to your users down the Kessel Run.

Number 4: Use a web server to serve your static resources. Sometimes a CDN is overkill. You can gain similar benefits by configuring your system so that all static content (images, CSS, JavaScript, etc.) is served by your web server instead of by your application server. It is well known that a highly optimized web server can serve static resources a lot faster than an application server can. You can use your proxy configuration and two Liferay properties to let your faster web server send these to the browser, leaving Liferay and your application server to only have to worry about the dynamic pieces of the request.

Liferay Configuration Step 1: Set the following property in your portal-ext.properties file: theme.virtual.path=/var/www/themes

Step 2: Set the following property in your theme: /themes/beautiful-day-theme

Step 3: Set your server proxy to exclude from the proxy the path to the theme. This will vary from web server to web server. For Apache and mod_proxy, you would add this to your configuration file: ProxyPass /themes !

With this configuration, Liferay will deploy your theme to the path specified, which will be served up by your web server.

Number 3: Put some of that Comcast Speed Boost into the server.

Number 3: CSS/JS Sprites You heard it here: programmers are not lazy. When anybody is under a tight deadline, it's faster to get the project done if you implement it using experience already under your belt. If, however, you take the time to learn to use some of Liferay's built-in tag libraries, the performance benefits will pay off.

Instead of standard tags, use the tag as shown above.

What does this do? What's faster, transferring 100KB over 1 HTTP connection or opening up 10 connections for 10KB each? This is the reason developers have moved to CSS sprites for graphics. If you use the Liferay tag libraries, we will do all the packing and imaging for you. Upon deployment, Liferay, using the StripFilter and MinifierFilter, will automatically create a .sprite.png and .sprite.gif (for any IE 6 users out there), and generate code in the pages that looks like this:

Less work, same performance benefit We don't force you to cut up images. If you have 50 icons on one page, we consolidate that into one file automatically. The filters understand CSS too.

Number 2: Buy another server from the planet Krypton.

Number 2: Stupid Database tricks. Trick 1: Read-writer Database This allows you to direct write operations and read operations to separate data sources. You must configure your database for replication in order to do this. All major databases support this.

jdbc.read.driverClassName=com.mysql.jdbc.Driver jdbc.read.url=jdbc:mysql://dbread.com/lportal?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false jdbc.read.username= jdbc.read.password=

jdbc.write.driverClassName=com.mysql.jdbc.Driver jdbc.write.url=jdbc:mysql://dbwrite.com/lportal?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=fal jdbc.write.username= jdbc.write.password=

Make sure the spring config is included in your portal-ext.properties file (see next slide)

Read Writer Database spring.configs=\ META-INF/base-spring.xml,\ \ META-INF/hibernate-spring.xml,\ META-INF/infrastructure-spring.xml,\ META-INF/management-spring.xml,\ \ META-INF/util-spring.xml,\ \ META-INF/editor-spring.xml,\ META-INF/jcr-spring.xml,\ META-INF/messaging-spring.xml,\ META-INF/scheduler-spring.xml,\ META-INF/search-spring.xml,\ \ META-INF/counter-spring.xml,\ META-INF/document-library-spring.xml,\ META-INF/lock-spring.xml,\ META-INF/mail-spring.xml,\ META-INF/portal-spring.xml,\ META-INF/portlet-container-spring.xml,\ META-INF/wsrp-spring.xml,\ \ META-INF/mirage-spring.xml,\ \ META-INF/dynamic-data-source-spring.xml,\ #META-INF/shard-data-source-spring.xml,\ \ META-INF/ext-spring.xml

You will now have a dedicated data source where write requests will go. With replication enabled, updates to all nodes can be done much faster by your database software. You can have one configuration of your database optimized for reads. You can have one configuration of your database optimized for writes.

Trick #2: Database Sharding Sharding is splitting up your database by various types of data that may be in it. It is a technique used for high scalability scenarios. One algorithm might be to split up your users: –

A-D: Database 1

–

E-H: Database 2

–

(etc)

When users log in, they are directed to the instance of the app that has their data in it.

Liferay Sharding Liferay supports sharding through portal instances. You can create separate portal instances with your application in them, enable sharding, and Liferay will use its round robin shard selector to determine where users should go. To enable sharding, use your portal-ext.properties file: shard.selector=com.liferay.portal.dao.shard.RoundRobinShardSelector

More Sharding spring.configs=\ META-INF/base-spring.xml,\ \ META-INF/hibernate-spring.xml,\ META-INF/infrastructure-spring.xml,\ META-INF/management-spring.xml,\ \ META-INF/util-spring.xml,\ \ META-INF/editor-spring.xml,\ META-INF/jcr-spring.xml,\ META-INF/messaging-spring.xml,\ META-INF/scheduler-spring.xml,\ META-INF/search-spring.xml,\ \ META-INF/counter-spring.xml,\ META-INF/document-library-spring.xml,\ META-INF/lock-spring.xml,\ META-INF/mail-spring.xml,\ META-INF/portal-spring.xml,\ META-INF/portlet-container-spring.xml,\ META-INF/wsrp-spring.xml,\ \ META-INF/mirage-spring.xml,\ \ #META-INF/dynamic-data-source-spring.xml,\ META-INF/shard-data-source-spring.xml,\ \

And, of course, enable it in your spring configs.

Number 1: Red Bull anyone?

Number 1: HTML Positioning of Elements Here's a code snippet from Yahoo.com. Anybody notice anything strange? if(window.yzq_p==null) document.write("");

Positioning Changes Things Bingo! if(window.yzq_p==null) document.write("");

They have JavaScript after the HTML tag. That's not according to spec, but all the browsers support it, and it improves performance.

JavaScript in Portlets If you have JavaScript in your portlets, you can control where Liferay positions it via the liferay-portlet.xml file. /html/portlet/message_boards/javascript.js

Here's your algorithm: try it in the footer first. If you have errors, put it back in the header. This is just as simple: /html/portlet/message_boards/javascript.js

And there you have it!