The Idea

ExtDoc’s approach is to adopt the way Eclipse displays Javadoc, i.e. in a view or when hovering code elements, and to create a framework which allows to contribute any information to similar displays. For instance, Code Recommenders holds statistical information of many kinds for many popular frameworks, but currently lacks a way of displaying all in one place. In order to have information displayed with ExtDoc, each information contributor simply has to implement a “provider” through which he is asked to submit his data for the current user selection. ExtDoc then puts together all providers and allows the user to navigate through the information in multiple spots.

The ExtDoc View

Just like the console or the Javadoc view, the ExtDoc view can be located somewhere around the main editor and reacts to any selection in the editor. In the following, all available providers are depicted as they appear in the ExtDoc view. Additional to the providers’ contents, which are displayed as one scrollable “document” on the right, there is a table on the left indicating all available providers and their status. For example, providers are grayed out if they have nothing available for the current selection or are disabled by the user.

The table also offers the possibility to change the provider order by drag and drop and to disable/re-enable providers perminately. Finally, on top of the table the selection’s location type is displayed (e.g. “field declaration”), since some providers distinct between up to 9 location types. For example, a user might find different information useful when selecting a type in an “extends” statement than in a method body. Therefore the user’s selection of providers is also location-sensitive.

Currently available Providers

The method calls provider offers information on which methods should typically be called for the selected Java element. For example, when selecting SWT’s Text type, it displays the most frequently called methods. When selecting a local variable of type Text, it even takes into consideration which methods have already been called on the variable, which usually influences the probabilities of further method calls.

Sometimes even whole blocks frequently occur in different source codes, e.g. the configuration of reoccurring elements. As a result, not only single method calls but also whole code examples are proposed by ExtDoc. Note: this provider is currently in development and only displays information for a few selected elements.

The usage of a framework typically involves extending provided interfaces, abstract classes or even concrete classes. Not seldom, one is expected to override already implemented methods in order to achieve certain tasks and therefore has to understand the superclass(es) code or rely on a comprehensive documentation. Code Recommenders computes which methods of a class typically are overriden, so-called subclassing directives, which are also displayed in ExtDoc. When selecting an overriden method, ExtDoc even displays which of the superclass’ methods are typically called from the method implementation, e.g. for configuration purposes.

Often framework objects are such comprehensive that the set of overriden methods differs between tasks, i.e. one method might be required to be specified in one scenario, but of no interest at all in the other. Therefore also subclassing patterns are mined, as depicted in the following screenshot.

Finally, the social bookmarks provider allows the users to share web resources. For example, when one experiences difficulties with an API element, he might submit a solution he found one the web as a reference for all other users that might come along this element and have similar issues. In order to filter the most relevant resources, the community is able to rate each link.

Community Feedback

As depicted in all screenshots, all current providers allow users to rate their content through a 5-star system, either the displayed element information as a whole, or single items such as a specific patterns. Furthermore, there is a standard ExtDoc widget which allows commenting the provider content. The following screenshot depicts the expanded comments section underneath the subclassing patterns provider.

The ExtDoc community widgets allow both, communication between the users, e.g. additional information to the displayed data, and feedback to the provider author.

Further ExtDoc Locations

All above screenshots displayed the providers as they appear in the ExtDoc view. However, similar display is also available in two additional locations in which users typically look for documentation. First, the Javadoc pop-up, which is displayed when hovering an element in the editor, is replaced with an ExtDoc pop-up. It contains the same providers “document” as the view does and an additional toolbar underneath, which allows jumping/auto-scrolling to the specific provider. Now, there is no more Javadoc available!? Don’t worry, ExtDoc also contains a provider displaying the traditional Javadoc, as depicted in the very first screenshot :-)

The third location for ExtDoc is the documentation of elements selected in the code assistant pop-up. It offers the same features as the hover pop-up does, as shown below:

Obtaining ExtDoc

Since ExtDoc, among some other Code Recommenders features, is in the testing phase for an upcoming drop, you have to access it either through the head update site or the ExtDoc update site. Both links are given below:

http://www.eclipse.org/recommenders/download/
http://vandyk.st.informatik.tu-darmstadt.de/updates/extdoc/

Feedback on the current state as well as suggestions for further development are much welcome :-)

The following contains conceptual updates introduced with the first prototype already available for download.

The New ExtDoc View

The most important user interface update is the relocation of the providers as well as their content. In the first prototypes each provider had a separate tab, now all content is listed in a single view, as seen below:

This design has multiple advantages over a separation by tabs. The user can easily scroll through all information, i.e. you don’t have to click multiple places to access all available information and you might also maximize the window to have everything at eyesight. Additionally, no space is wasted, e.g. by single providers which would not fill the whole view. Furthermore, you don’t have to skip grayed out tabs in case of providers having no content for the current selection. Also configuration is much easier (see below).

Providers Control

The second important update is the provider list on the left of the view. Not only does this provide a better visualization of available providers, but also options to configure them. The following illustrates multiple states:

A provider in black font indicates content displayed in the main view, grayed out providers do not contribute. This is the case when they are active but don’t have information available (indicated by the blue selection box); or they are disabled by the user (the selection box isn’t ticked). Disabling/enabling instantly leads to content being removed or added on the right. You are also able to change the order in which provider content is displayed by drag and drop of items in the provider list. The selection of providers as well as their order is constantly stored.

Location-Based Provider Sets

Especially when the amount of content providers grows (several more are in development) you won’t always be interested in displaying all. But, what might be irrelevant now might be required with the next click in the editor.

We consider a distinction between location types, as described in an earlier post. For example, you might not be interested in how to subclass a type if it’s just the type of a variable you defined. You are rather interested in how to use it. Therefore the provider selection is stored per location type. What you select for one location (e.g. a field declaration) is not activated for another location (e.g. a method declaration) until it’s also selected there.

In the next days the current version will be improved in terms of design and stability. Then a further unofficial prototype will be released for download. First testers as well as any feedback is much welcome :-)

Documentation in Eclipse/JDT – the Javadoc – however is context-independent. There is at most one static text assigned to a type or method, which will be displayed no matter what context it is used in. This solution is obvious since usually there is only general information provided by the API designer. But when working with large amount of data mined from different contexts in order to automatically infer documentation, as the Eclipse Code Recommenders project does, it will become reasonable to apply a context-dependent filter on this data as well.

The Extended Documentation Platform (ExtDoc), an upcoming feature of Code Recommenders, will host multiple documentation providers which require several information to identify the user’s current interest – such as the location of a Java element he selects. To give a better idea of the capabilities of location resolval, in the following different locations inside Java code will be discussed first. Then the usage of location information is illustrated by the example of one of the documentation providers which will be included in the ExtDoc feature.

Java Element Locations

The type is probably one of the most diversely used concepts in Java. This is due to the several roles a type can take just by different syntactical usage. The same type name in a single source code might indicate the variable type in one occasion and be the constructor identifier in the next. The following code snippet gives an example of different type usages within just a few lines of code – which could easily be extended, e.g. by type parameters. In each occasion the type has a different role within the class and different documentation might be relevant.

Discrimination between locations can also be applied to methods and variables. And it can be of high value to the documentation provider as well. Assuming that exemplary code snippets for a selected element are to be displayed, relevant examples for a variable in a field declaration might differ very much from relevant examples for the same field variable used inside a method, where methods might have already been invoked on the variable. Also a variable passed as a method argument could have other examples presented for since its common usage might differ, e.g. only getters might be invoked. A case study for using location information is given below.

Case Study: Subclassing Directives

Subclassing – the inheritance and extension/overloading of a type – is a powerful concept for software architects, but can also lead to a high complexity. Usually your IDE will show you all inherited methods and fields; to identify their usage however is up to you. This can lead to problems, especially if you are required to call certain methods (e.g. for configuration) or even override methods that are not declared as abstract by the superclass. One way to handle this is to read through the whole Javadoc – or to use subclassing directives mined from actual API usage!

Subclassing directives are mined and provided by the Code Recommenders project; one of the first ExtDoc providers will display them. The following image depicts a UI prototype for a subclassing documentation for the JFace Wizard type. Each line indicates in how many cases methods of Wizard are overridden by the subclass. Furthermore the methods of Wizard usually called from within subclasses will be displayed.

It is obvious that you will want to have this information displayed when you are extending the Wizard type, e.g. when clicking on “Wizard” in the “extends” section of your class. Also the constructor might be a good place to learn more about the extension of the class. However, it would be quite pointless to display such information when Wizard is just the type of method’s parameter – you won’t be able to subclass the parameter anyway. Instead you will rather be interest in call patterns, i.e. which methods usually are invoked on Wizard. This also applies to other occasions of variable declarations and type usage, e.g. the method return type.

Another perspective on subclassing arises from single methods: Should an overridden method call its super implementation? And which other methods are expected to be called from within this particular method? The following is a UI prototype for subclassing directives on the method level:

Again the relevance of this information differs significantly between different locations. As the information about methods usually called from within the new implementation is only relevant to be displayed when actually working “inside” the method, it would be confusing to have subclassing information displayed when the method is selected in the context of another method just referencing/calling it.

Conclusion

As a conclusion it can be of great benefit for the user to have a documentation system which tells apart the multiple different contexts and locations Java elements do appear in. Especially when dealing with large amounts of data the context is powerful information to discriminate between the relevance of information in order to just deliver documentation that is of interest to the user. The case study of a subclassing directives provider has given a first example of distinguishing between locations of a type. In upcoming articles further documentation providers, along with different aspects of using Java and Eclipse will be presented.

Listening to Selections

There is a Eclipse standard solution for listening to GUI selection events, which includes implementing the ISelectionListener interface. It will be notified on most selections inside editors and views (the IWorkbenchPart).

class MySelectionListener implements ISelectionListener {
	@Override
	public void selectionChanged(IWorkbenchPart part, ISelection selection) {
		...
	}
}

In order to register the listener, the current IWorkbenchWindow and its ISelectionService have to be accessed, which can be achieved using the three lines of code given below. Additional to the selection listener also a IPartListener is registered, which will be described later. The third line might result in a NullPointerException if it is called “too early”, e.g. when the plugin is started, because the window’s page (the container for all editors and views) might not yet be initialized. Therefore this listener should be registered after the plugin loaded. A safer way is, when providing a new window element like a view, to call getSite().getPage() from the UI part extension.

IWorkbenchWindow window = PlatformUI.getWorkbench().getActiveWorkbenchWindow();
window.getSelectionService().addSelectionListener(new MySelectionListener());
window.getActivePage().addPartListener(new MyPartListener());

A major problem with tracking Java element selections using the ISelectionListener is that it does not listen to all selections in editors. In fact, only marking and de-marking of text is tracked. In order to implement additional listeners tracking cursor position changes, the specific editor has to be accessed first. As there is no single “static” element like the workbench window, but instead each editor window/tab one opens has its own listener service, newly opened editors have to be detected using an IPartListener. Since all single windows (editors and views) are tracked by the IPartListener, all parts except editors – Java editors in particular- have to be filtered.

class PartListener implements IPartListener {
	@Override
	public void partOpened(final IWorkbenchPart part) {
		if (workbenchPart instanceof JavaEditor) {
			final JavaEditor editor = (JavaEditor) workbenchPart;
			final StyledText text = (StyledText) editor.getAdapter(Control.class);
			text.addKeyListener(this.myCursorListener);
			text.addMouseListener(this.myCursorListener);
		}
	}
}

Having obtained access to a Java editor, one can add listeners for mouse- and key events to become aware of every cursor position change in any opened Java editor. Notice that, as we now have our own listener for Java editors, there is no need to fire selection events using the ISelectionListener, so one might want to add
if(!(part instanceof JavaEditor)){...} to his/her implementation of selectionChanged.

class MyCursorListener implements MouseListener, KeyListener {
	@Override
	public void keyPressed(final KeyEvent event) {
		...
	}
	@Override
	public void mouseUp(final KeyEvent event) {
		...
	}
}

Resolving the Selection

Being aware of selection events, one finally wants to know what exactly is selected by the user. The ISelectionListener already provides the IWorkbenchPart and the ISelection. However, the cursor listener does not. This is not much of a problem since those information can generally be accessed from any place at any time – assuming that there is an active IWorkbenchPage (see above).

final IWorkbenchPage activePage = PlatformUI.getWorkbench().getActiveWorkbenchWindow().getActivePage();
IWorkbenchPart part = activePage.getActivePart();
ISelection selection = activePage.getSelection());

An ISelection will be one of IBlockTextSelection, IMarkSelection, IStructuredSelection, ITextSelection or ITreeSelection, where each selection type contains some specific information, e.g. the selected elements for a tree selection. However, we are interested in further information not available from the selection information. What we want in particular is indicated by the following interface.

public interface IJavaElementSelection {
	IJavaElement getJavaElement();
	JavaElementLocation getElementLocation();
	JavaContentAssistInvocationContext getInvocationContext();
	ASTNode getAstNode();
}

The ASTNode and the JavaElementLocation are subject of an upcoming article. The IJavaElement (representing nearly everything in JDT – from projects to variables) and the JavaContentAssistInvocationContext (which is also created when invoking the content assist using ctrl+space and contains much useful information) have to be resolved depending on the selection type. The following method takes a workbench part and a selection as obtained above and resolves the required information for tree selections (package explorer, content outline, …) and selections in Java editors.

public IJavaElementSelection resolve(IWorkbenchPart part, ISelection selection) {
	IJavaElementSelection context = null;
	if (selection instanceof ITreeSelection) {
		context = fromTreeSelection((ITreeSelection) selection);
	} else if (part instanceof JavaEditor && selection instanceof ITextSelection){
		context = fromEditor((JavaEditor) part, (ITextSelection) selection);
	}
	return context;
}

Taking a Java element from a tree selection is fairly easy. We will assume that when several elements should be selected we are only interested in the first one as the ExtDoc Platform will only support single-element views. A JavaContentAssistInvocationContext can only be created for editors, so the Java element is enough for now.

private IJavaElementSelection fromTreeSelection(final ITreeSelection selection) {
	IJavaElement javaElement = null;
	final Object firstElement = selection.getFirstElement();
	if (firstElement instanceof IJavaElement) {
		javaElement = (IJavaElement) firstElement;
	}
	return new JavaElementSelection(javaElement);
}

Since an ITextSelection is an “abstract” scenario applicable to any text editor, one cannot access the selected Java element directly. Instead, several steps have to be taken, as shown in the following method.

private IJavaElementSelection resolveFromEditor(final JavaEditor editor, final ITextSelection selection) throws JavaModelException {
	IJavaElement javaElement = null;
	final IEditorInput input = editor.getEditorInput();
	final ITypeRoot root = (ITypeRoot) JavaUI.getEditorInputJavaElement(input);
	final IJavaElement[] elements = root.codeSelect(selection.getOffset(), 0);
	if (elements.length > 0) {
		javaElement = elements[0];
	}
	return new JavaElementSelection(javaElement, selection.getOffset(), editor);
}

Now a different constructor for a JavaElementSelection (our own type, see the interface above) is called, also containing the selection offset and a reference to the editor. This extra information is stored as we might later be interested in creating a JavaContentAssistInvocationContext for the identified Java element selection. This can be achieved comparatively easily:

new JavaContentAssistInvocationContext(editor.getViewer(), offset, editor);

Conclusion

As a conclusion one can say that the general Eclipse listener interface works well for most task of tracking user activity. However, since most of the effort is spend on tracking and resolving Java element selections in the Java editors, a JDT-specific selection listening service might be a relevant extension of the JDT API since one can image many different plugins built upon such information.

In an upcoming article the element selection information will be used to compute further context information, including the resolves abstract syntax tree (AST) node as well as the “code position” (is a variable name written as part of a field declaration, as a parameter declaration or just used inside a method block?) which is of interest to the Extended Documentation Platform as different information might be relevant for the same Java element used in unequal occasions.

Extended Views

The most important part of the ExtDoc Platform is its tabbed view, which will provide an independent page for each documentation provider. This means that despite being integrated into the same Eclipse view, each provider is free on how to present his content. To enable the provider to deliver information suiting the user’s focus the ExtDoc platform will inform him about any of the user’s new actions and selections inside the IDE.

As there will be many providers offered by Code Recommenders and also any third-party contributor is very much welcome to register his own documentation provider, the user will be given the possibility to choose which providers he wants to be displayed. Furthermore he will be able to define several ExtDoc views on his own to set up different collections of providers between which he might switch depending on the type of information he is currently interested in.

The following images show how different documentation might be integrated into the views. As mentioned, the provider is not limited to a specific kind of display (e.g. only text), instead he is free to compose his page using any SWT widgets. However, we decided to stick with the Javadoc appeal for our first prototypes.

Extended Pop-Ups

Other places Eclipse traditionally displays documentation (Javadoc) in include the pop-ups appearing when selecting or hovering over an element in the editor, as well as when selecting a completion proposal in the code assist. For such pop-ups the ExtDoc Platform will extend the original Javadoc with useful information obtained from its providers. Furthermore, it will again offer tabs to browse through the providers’ single pages. The composition of documentation providers for pop-ups will also, again, be modifiable by the user.

The first two images show general pop-ups, first unselected and then expanded by selection; the third image illustrates the content that is shown along a selected completion proposal.

Community Features

One of the core ideas of an “IDE 2.0″ is users sharing information. Currently Code Recommenders supports this implicitly by distributing data mined from actual API usage. To make user contribution explicit, the ExtDoc Platform will provide several features to enable users to share content and communicate about it. Such content might, for example, include changes to a element-specific Wiki or new code templates for a class type.

Ways of sharing content will be displayed in the next section. In order to communicate over content, features such as ratings and comments are offered by the ExtDoc platform. A potential rating feature is indicated by the stars in the previous images, the comments feature is illustrated in the first of the following two images. As a user might want to directly address the provider author, e.g. to suggest new features, a feedback framework will also be provided, as illustrated in the second image.

Sharing Content

As mentioned before, users will be able to contribute new content in various forms. Of course, the specific type of content depends on the documentation provider and which functionality he enables. The Code Recommenders providers will allow several types of content to be shared. For example, the definitions of new code patterns or subclassing templates. Another example is a community-driven Wiki-like Javadoc system.

As it applies to all aspects of the ExtDoc platform, each content provider is free in how to set-up his services, e.g. how to deal with user data, if user input should be enabled, which kind of input is entered in which way (e.g. an editor in the main perspective vs. dialog pop-ups) and how he proceeds with incoming data. However, resulting from the integration of several Code Recommenders providers, a variety of classes and interfaces will be provided to ease the set-up of content sharing systems.

The following two images show two different content editors as they might be implemented for Code Recommenders providers.

Settings

Eventually, the variety of API documentation and the possibilities to deal with it have to be handled by the user. With a growing number of documentation providers one of the most important user decisions is which to display. However, the user preferences may change quickly, e.g. for one class he might only require his default selection, for another he might want to use a special documentation “toolkit”. Therefore, as displayed in the first picture, the user will be able to define multiple views and for each which providers should appear in which order.

Furthermore, also the providers displayed in pop-ups can be selected, as depicted in the second image.

Some documentation providers might also offer specific user settings. Again, this option is supported by the ExtDoc platform but not required or limited in any way. A possible preferences page for one of the providers is shown in the last picture.

Summary

To sum up, the UI prototypes show that ExtDoc will impact the way users are looking for information while coding. Up until now the developer has to search the Internet as soon as the Javadoc, written by the API designer, is missing of insufficient. With ExtDoc and its providers the intelligence of the community will come right into the IDE. The appropriate documentation might just be a tab away …

The next article(s) will go much more into the details of which individual providers are planned to be released from the first versions of ExtDoc on. This will include a general improvement on the Javadoc view, subclassing directives, code patterns / templates, a Wiki and more. See you there!

So Marcel and I sat together to discuss a framework for content providers to integrate their information into the Eclipse IDE as efficient as possible. Finally, we’ve proposed our ideas as an Eclipse project to Google Summer of Code and got accepted! Thus, this blog post not only is a vision to our work, but an introduction for the Eclipse and GSOC communities as well.

Extended Documentation (ExtDoc)

For providing the user with very specific, context-sensitive information, Eclipse offers several extension points, e.g. to the editor’s code assist feature. Generally, however, most information is included in special views, for which – especially when presenting mainly text – a lot of design and implementation work is (re-)done by each plugin. Especially the Eclipse Code Recommenders project, with its tons of mined data regarding API usage, and its several sub-projects, has to deal with this issue when trying to provide the user with the most comprehensive documentation about the packages, classes, methods, etc. he wants to know more about.

To ease the integration into the IDE, the Code Recommenders ExtDoc framework will provide a single, tabbed view to which each provider can register and submit his content to. The user is then able to configure the view according to his current needs (e.g. which content to show) and easily browse through all available information. Furthermore, he will be able to interact with the content, share thoughts about it with the community, and provide feedback to the publisher – all through one view.

Additionally to the view, the provider content will also be available in the pop-ups that appear when hovering over elements in the editor, as well as when selecting, for example, a method in the completion proposals system. This will bring the information even closer to the user and will allow him to directly browse the provider content on the spot. Due to the size limitations of the pop-ups, the content provider will have the possibility to provider different content than for the view, for example to provide a summary or only the most important information. Once again, the standard Javadoc display will not be replaced, but enhanced with tabs

Documentation 2.0

Not only can the IDE be enriched by community-driven information, as done by the Recommenders project, the IDE can also be used to make API documentation “more 2.0″ and to give feedback to  the publishers. To do so one important feature of ExtDoc will be to add standard solutions for interacting with provider content. First, depending on the content, the user will be able to add, modify and remove items, for example a code template. Furthermore, he will be able to communicate about the single items through ratings and a comments system. Finally, if desired, he can also provide direct feedback to the publisher, e.g. if he has a feature request.

ExtDoc for Developers

For the (third-party) providers of content, the goal of ExtDoc is to be as supportive and at the same time as flexible as possible. A lot of special interfaces, classes, methods, SWT components, design templates, etc. shipped with the ExtDoc framework can be deployed by your provider, but in principle the view is yours! Usually a plugin provides a SWT composite in order to define a view and so will ExtDoc ask for SWT composites from its providers. Thus you will be free in how to build your view and might only reuse certain components, e.g. the visual rating system. Also for other parts, like the feedback form, the ExtDoc framework will provide everything, from simple builders to absolute freedom. Even for dealing with user content we will offer you solutions, for example a client/server implementation for the comments, but in the end it’s up to you and your provider extension which features to make available and how to deal with the data.

One further thing to notice is the context information we will provide. The Recommenders project already does a great job in augmenting the default information given by the Eclipse API, for example by parsing ASTs. All this information is combined and delivered to each content provider as soon as the user changes his selection in the IDE. By doing so the providers’ efforts in analyzing the context is reduced and they can decide on their own in which occasions their displayed content has to change in which way.

Upcoming

Despite the detailed information given in some sections, this vision should not have been more than an introduction to the topic. As the project evolves we will publish further information regarding the single aspects. As a next step, I will submit a series of GUI prototypes to concretize the ideas given here and give you the opportunity to help shaping this interesting project according to your needs and wishes.