Spray Developer Guide

Spray Developer Guide

Spray – A quick way of creating Graphiti Developer Guide Authors: Jos Warmer, Karsten Thoms, Joerg Reichert Contents 1. Spray - A quick way of creating Graphiti 1. Target Runtime Framework 2. Reference Implementation 3. Framework Development 4. Language Development 5. Code Generation 6. Testing 7. User Interface 2. Setting up the developer environment 1. Source Control 2. Developer IDE 1. IDE 2. Update Sites 3. Features to Install 4. Troubleshooting 5. Add Spray Git repository to Eclipse 6. Add Spray task repository to Eclipse 7. Add Spray Jenkins connector to Eclipse 3. Target Platform 4. Workspace Setup 3. The Spray DSL Infrastructure 4. Spray Generator 1. Reference Implementation 2. Indentation 3. Import Organizing 4. Modularity 1. Template Methods 2. Hook Templates 3. Generated Code 5. Code Documentation 1. Overridden Methods 5. Graphiti 6. Project Setup 1. Naming 2. Source Management 3. Adding a new Plugin project

1. Manifest 2. build.properties 3. Project Specific Settings 4. Build 4. Dependency Management 1. Imported Packages 5. Plugin Development 1. plugin.xml 6. Spray Maven Build 7. Parent POM 1. Plugin Management 8. Profiles 1. Profile @skip-ui-tests@ 7. Continuous Integration 1. Introduction 2. Build Jobs 1. Spray CI Build 2. Spray Experimental Build 3. Administration 8. Release Process 9. Testing 1. Test Projects 2. Test aspects 1. Testing using the Spray DSL Editor enviroment 2. Testing the generated Graphiti editor 3. Test approaches 1. JUnit tests 2. Mocking 3. Xtext tests 4. JUnit plug-in tests 5. UI tests 6. Measuring test coverage 7. Test data organization 8. Generator template tests 4. Spray Language Tests 5. Runtime Tests 10.Issue Tracking 1. Tracking System 2. Git Commit Message

General rules for designing Spray Target Runtime Framework The target graphical modeling framework of Spray is Graphiti. Though other runtimes might be possible to describe with the DSLs, this is not the primary goal.

Reference Implementation All proposed features that should be covered by code generation must be also covered by Reference Implementation code. This reference code must be developed in a way that it can be fully generated from Reference Models. The quality of this reference code is crucial.

Framework Development Although Graphiti has already a strong API it will be necessary to create framework classes also for Spray. The Spray framework API should be developed in a way that it will be potentially proposed to Graphiti as enhancement requests. The framework layer of Spray should be as small as possible and as large as required.

Language Development The DSLs developed in Spray should follow best practices in Language Design. • • • • •

Keep it simple: creating a simple Graphiti editor should be straightforward and quick to do More advanced features may not break the previous rule. They should be additive in the sense that you won’t see them if yo don’t need them. The Spray DSL does not need to enable everything possible with Graphiti. For advanced features, users will always be able to directly use the Graphiti API and combine such handwritten code with generated code. If a language is hard to parse it is also likely hard to understand. Separation of concerns: If different concerns must be described in a DSL, it might require different DSLs

Code Generation • • • •

Code Generator is designed to enable mixing generated code with handwritten code without losing the possibility for code generation. Code Generator uses the generation gap pattern to give maximum flexibility to mix handwritten code with generated code. The Code Generator must be executable standalone (Maven, Ant, Commandline) without an Eclipse installation The Code Generator should be extensible by the user without the need to change the original plugin sources

Testing All features of the DSLs will be covered by models that will be tested by unit tests.

User Interface •

The UI should help the user to do common actions

Setting up the developer environment Source Control We are using Git as source management system. The primary repository location is: https://code.google.com/a/ eclipselabs.org/p/spray/ See http://code.google.com/a/eclipselabs.org/p/spray/source/checkout for details. You can new clone Git from the command line or use Eclipse, which is explained in a later section of this guide.

Developer IDE IDE Download latest Eclipse Indigo Classic appropriate for your platform: http://www.eclipse.org/downloads/packages/ eclipse-classic-371/indigosr1

Update Sites The update sites used to install the features required to develop Spray can be imported in your IDE. As a base take any Eclipse 3.7 distribution. Then open workspace preferences Install / Update -> Available Software Site and click the Import button. When you already cloned the Spray repository, then choose the file /org.eclipselabs.spray.devenv/ updatesites-spray-dev.xml otherwise import the file from the CI server: https://spray.ci.cloudbees.com/job/spray-ci-build/lastSuccessfulBuild/artifact/ devtools/org.eclipselabs.spray.devenv/updatesites-spray-dev.xml This xml file will add all required update sites to visit later when installing the plug-ins for the development environment.

Features to Install Choose now Help -> Install New Software. Choose the following update site in „Work with”: Spray Development Environment, this will install • • • • • • • • • • • • •

MWE SDK 1.2.1 MWE 2 language SDK 2.2.0 MWE 2 runtime SDK 2.2.0 EMF – Eclipse Modeling Framework SDK 2.7.0 Graphiti SDK 0.8.1 Xtext Antlr 2.0.0 SWT Bot xtext-utils unittesting Xtext SDK 2.2.0 EclEmma Java Code Coverage 1.5.3 Mylyn 3.6.2 EGit 1.1.0 Zest SDK 1.3.0

Troubleshooting If you have already installed one of the above mentioned features you might experience problems installing the newer features. In this case uninstall the features in question before (Eclipse / About Eclipse / Installation Details / Installed Software -> Uninstall).

For example, if you need to upgrade Xtext, uninstall all Xtext features, and the install Xtext SDK again with Install New Software.

Add Spray Git repository to Eclipse You can use our predefined team project set file to get the Spray projects into your workspace: • •

File > Import... > Team/Team Project Set, Next File: /devtools/org.eclipselabs.spray.devenv/projectSet.psf, Finish

This will clone the Spray repository into your Eclipse workspace folder, create a number of working sets and add the projects to the working sets. To see the working sets go to the „View menu” icon of the Package Explorer and select Working Sets as Top Level Elements. If you want to specify the location where to clone the Spray repository locally do the following: • • • •

File > Git/Projects from Git Add , Next Search, OK

If you had cloned the Spray repository locally and want to use that repository: • • •

• •

File > Git/Projects from Git Clone Paste https://code.google.com/a/eclipselabs.org/p/spray/ in the URI field and add your Google Account user name and your GoogleCode.com password (if your are logged in at the Spray Google Code site, click the Profiles link in the upper right corner and then go to the Settings tab) , Next select all branches, Next select your local path to store the Git repository clone, Finish

If you want to apply the Spray workset configuration without being forced to clone the Spray repository again, you can copy devtools/org.eclipselabs.spray.devenv/workingsets.xml to /.plugins/ org.eclipse.ui.workbench/workingsets.xml

Add Spray task repository to Eclipse • • •

• • •

Window > Show View > Other... > Mylyn/Task Repositories Open context menu in Task Repositories view > Add task repositories... Google Code, Next • URL: Paste http://code.google.com/a/eclipselabs.org/p/spray/ • Label: Spray • unselect Anonymous, if you later want to push code • enter your Google Account credentials (now use as password your Google Mail password, not the Google Code Password), Finish Add New Query use predefined query > Open issues Window > Show View > Other... > Mylyn/Task List

Add Spray Jenkins connector to Eclipse • •

Window > Show View > Other... > Mylyn/Builds Click on the first of the icons in the right of the view header > Hudson (supports Jenkins), Next • Server: https://spray.ci.cloudbees.com • Label: Spray • select Anonymous

•

• click on Refresh button • select job "spray-ci-build" • Finish click on spray-ci-build to show the current job status

Target Platform We are targeting for Eclipse Indigo. A target platform definition is provided in releng/ org.eclipselabs.spray.targetplatform/spray.target. To set this target platform: • • •

Import the project from path releng/org.eclipselabs.spray.targetplatform Open Workspace properties, go to Plug-in Development / Target Platform Here you should see Spray Target Platform. Check it and press Apply.

The target platform contains the following features

• • • • • • • • •

Eclipse Platform SDK 3.7.1 - http://download.eclipse.org/eclipse/updates/3.7 EMF – Eclipse Modeling Framework SDK 2.7.0 - http://download.eclipse.org/releases/indigo Graphiti SDK 0.8.1 - http://download.eclipse.org/graphiti/updates/0.8.0 MWE SDK 1.2.1 - http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/ Xtext SDK 2.2.0 - http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/ Xtend SDK 2.2.0 - http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/ Xtext Antlr 2.0.0 - http://download.itemis.com/updates/releases/2.0.0/ xtext-utils unittesting - http://xtext-utils.eclipselabs.org.codespot.com/git.distribution/releases/unittesting-0.9.x SWT Bot - http://download.eclipse.org/technology/swtbot/helios/dev-build/update-site

Distributions with Eclipse Indigo + Xtext 2.0 can be found here: http://download.itemis.com/distros/

Workspace Setup When creating a new workspace make sure to set the following settings: Resource encoding: UTF-8. Project settings are stored in the devtools/org.eclipselabs.spray.devenv> project. Whenever possible, project specific settings should be applied and the settings file checked in. In org.eclipselabs.spray.devenv there is a Java formatter configuration formatter.xml used in this project. Apply this formatter before committing anything.

The Spray DSL Infrastructure This secting will contain information about • • • • • •

mapping to the DSL concepts to Graphiti concepts implementation of expressions scoping nodel interferer usage of injection etc.

Spray Generator Explanation of the Spray Xtend2 templates implementation and design decisions (architecture), how does (different) models and their combination with the templates map to the generated artifacts (what should be the expected output)

Reference Implementation All proposed features that should be covered by code generation must be also covered by Reference Implementation code. This reference code must be developed in a way that it can be fully generated from Reference Models. The quality of this reference code is crucial.

Indentation All Xtend templates should use 4 spaces instead of tabs. Please check your template Xtend code for occurances of the tab character. You can visualize them in your workspace by setting the preference option General / Editors / Text Editors / Show whitespace characters. To replace all tabs in the current file open the Find/Replace dialog and enter (values without the ' character, this is just used here to visualize the whitespaces) • • •

Find: '\t' Replace with: ' ' Check option "Regular expressions"

Import Organizing Import statements that must be computed from classes used within generated code must be evaluated before actually printing out the class. To do so, we use a similar pattern like in Xtext’s Domainmodel Example: Use an Import organizer, evaluate the class body with all qualified type names, and after evaluating it print out the class header with the collected dynamic imports and the body afterwards. This requires the following pattern in the Xtend classes: • •

Derive template class from FileGenerator Inject extension NamingExtensions

class DiagramTypeProvider extends FileGenerator @Inject extension NamingExtensions naming

•

{

Implement mainFile() with the following steps: • Print class header and package statement • Add static imports • Add // MARKER_IMPORT. This will denote the position where dynamic imports will inserted. • Print the class body content def mainFile(Diagram diagram, String className) ''' ¬´header(this)¬ª package ¬´diagram_package()¬ª; import org.eclipse.graphiti.dt.AbstractDiagramTypeProvider;

import org.eclipse.graphiti.tb.IToolBehaviorProvider; // MARKER_IMPORT «body» '''

•

Use the extension function shortName on any qualified name. It will print out the simple name of the qualified class name and collects the qualified name for the import manager.

«metaClass.javaInterfaceName.shortName» // javaInterfaceName computes the qualified class name of an EClass interface // prints out 'IFeatureProvider' "org.eclipse.graphiti.features.IFeatureProvider".shortName

Modularity Template Methods The templates should be organized into reasonable small template definition methods. Usually a good separation is to provide a template for each generated method, which are called when generating a class body. def mainFile (MetaClass metaClass, String className) ''' ... public class «className» extends AbstractCreateFeature { ... «generate_canCreate(metaClass)» «generate_create(metaClass)» «generate_createModelElement(metaClass)» «generate_getCreateImageId(metaClass)» «generate_hasDoneChanges(metaClass)» «generate_canUndo(metaClass)» } ''' def generate_canCreate (MetaClass metaClass) ''' «overrideHeader()» public boolean canCreate(ICreateContext context) { ... } '''

When the implementation of a template method grows too large, or is separatable into a sequence of logical parts, the template should delegate to subtemplate methods again. This makes the template code more maintainable, and allows users later to override smaller parts of the generator if customization is needed.

Hook Templates To allow insertion of additional fields, constants, methods etc. in (Java) class bodies by subclassed templates, the base class FileGenerator offers the hook template methods generate_additionalFields() and

generate_additionalMethods(). These methods should be called in a class body template. The user then can override these methods to insert additional code without the need to override the class template itself. def mainFile(MetaReference reference, String className) ''' ... public abstract class «className» extends AbstractUpdateFeature { «generate_additionalFields(reference)» // call hook template public «className»(IFeatureProvider fp) { super(fp); } «generate_canUpdate(reference)» ... «generate_additionalMethods(reference)» // call hook template } '''

Generated Code Also generated methods should not grow too large. The user should be able to override smaller methods in the extensions files. To be able to do this, the generated methods in base classes should mostly be public or protected, and not static nor final. When appropriate, callback methods should be generated where it is likely for the user to make changes without the need to override a complete method. For example, when creating a connection, the user might want to decorate it. Example (AddReferenceAsConnection.xtend): def mainFile(MetaReference reference, String className) ''' ... public class «className» extends AbstractAddFeature { ... «generate_add(reference)» // generate callback method «generate_decorateConnection(reference)» } ''' def generate_add (MetaReference reference) ''' ... public PictogramElement add(IAddContext context) { ... // call callback method in generated code decorateConnection (addConContext, connection); } ''' // template for callback method def generate_decorateConnection (MetaReference reference) ''' /** * Override this method to decorate the created connection */ protected void decorateConnection (IAddConnectionContext context, Connection connection) {} '''

Code Documentation Overridden Methods If a method from a base class is overridden by generated code, the method should declare the \@Override annotation and the {@inheritDoc} Javadoc annotation. To shorten this, «overrideHeader» generates the necessary code. Xtend Template: «overrideHeader» public void myMethod (...) Generated code: /** * {@inheritDoc} */ @Override public void myMethod (...)

Graphiti This secting will contain information about • • • •

the generated Graphiti code additional static platform artifacts architecture concepts of how to make the generated code adaptable

Project Setup Things to consider when developing Spray projects.

Naming Base package / Bundle ID Prefix: org.ecliselabs.spray

Source Management • •

•

Generated sources are not checked in The following resources must be added to .gitignore for each plugin • src-gen • target • bin • plugin.xml_gen Experimental features must be added on a branch until stabilized. This branch may be shared on the remote repository if it is of interest of other developers to contribute or review the changes.

Adding a new Plugin project Manifest •

Add a plugin.properties

# /** # * # * Copyright (c) 2011 The Spray Project. # * All rights reserved. This program and the accompanying materials # * are made available under the terms of the Eclipse Public License v1.0 # * which accompanies this distribution, and is available at # * http://www.eclipse.org/legal/epl-v10.html # * # * Contributors: # * Spray Project Team # * # */ # NLS_MESSAGEFORMAT_VAR pluginName = ADD DESCRIPTION providerName = Eclipselabs Spray •

Open MANIFEST.MF, change/add entries (exchange 0.3.0 by current development version from other plugins)

Bundle-Name: %pluginName Bundle-Vendor: %providerName Bundle-Version: 0.3.0.qualifier Bundle-Localization: plugin

build.properties Include the following files/directories, if they exist •

Binary Build • plugin.properties • feature.properties

•

• icons/ • images/ Source Build • pom.xml • launch/ • */*.launch • log4j.properties

Project Specific Settings To avoid confusion between different platforms and workspaces common settings should be defined as project specific settings. Those settings are checked in and thus shared. • • •

•

Resource: Text file encoding: Other / UTF-8 Resource: New text file delimiter: Other / Unix Java Code Style -> Formatter: • Check "Enable project specific settings" • select Active Profile "`spray_eclipse_formatter`" Java Editor -> Save Actions • Check "Enable project specific settings" • Check "Perform the selected actions on save" • Check „Format source code” / "Format all lines" • Check "Organize imports"

Build • •

Copy the pom.xml from org.eclipselabs.spray.generator.graphiti. Exchange the plugin’s name in org.eclipselabs.spray.generator.graphiti by the project’s name. Open releng/org.eclipselabs.spray.distribution/pom.xml and add the plugin as additional module

Dependency Management Imported Packages Require Bundle is sometimes a too tight dependency binding. It does not allow to exchange the implementing bundle, which sometimes might be desired. The following packages should imported instead of adding its bundles to the Required Bundles: • • • • •

org.eclipse.xtext.xbase.lib org.eclipse.xtext.xtend2.lib org.apache.log4j org.apache.commons.logging com.ibm.icu

Plugin Development plugin.xml Xtext generates a plugin.xml just once, on second generator execution it creates a plugin.xml_gen. This is because Xtext’s generator cannot merge changes into the plugin.xml and manual modifications might be intended. Differences between both files must be merged manually. The plugin.xml file is quite large and it can become difficult to decide, which differences are produced by the generator and which are manual changes. In order to identify the intended changes these places must be marked in the code by comments .... Example: