Supporting Multiple Resource types with EMF Editor

The resource model is one of those aspects of the Eclipse Modeling Framework that is extremely powerful yet can seem way too complicated for us mere mortals to muck with. "But what if I break something?" The thing about powerful abstractions is that the more powerful they are, the more they do; and the more they do, the more systems they touch and -- somehow -- the more mysterious they become.  I'll propose a little law here which I am sure is not original: "any abstraction that is sufficiently powerful will be initially opaque". Just where are those levers we need to pull to get things to do what we want? As a highly leveraged framework, EMF is full of such abstractions. So how to get past that learning barrier?

Well, it takes some up front commitment, really, but there is a pretty simple heuristic to it: Decide the things you want to change, look through the generated code for where the things are that you might want to change, and experiment. Or wait for someone else to do the experimenting for you, which is the smartest strategy of all. Ergo, you must be smart, because you're reading this. So without further ado, let's take a look at how to do something extremely cool.

The basic scenario is this: there are a number of different data formats -- in EMF speak, "Resource Implementations" that might be appropriate for a given user and use case. By far the best example of this is dealing with the problem of data transparency vs. data efficiency. XML is (arguably) readable using any text editor, and it is definitely the format you want to be using when something goes wrong, for example a reference is bad or you have a problem with encoding. But memory footprint and load and save performance wise, it's a total dog. In contrast, Binary is the format you want to use when you have a lot of data and you need to use it, but good luck figuring out what's in it without your editor.

Here's something that I'll bet many people -- even those who have been using EMF for a long time -- don't know. I didn't. EMF has built in support for a very efficient Binary Resource model. But how can we use it? There is (not last time I checked anyway) no resource type entry in .genmodel for "Binary".

In fact, it turns out that it is ridiculously easy to replace the generated XML resource support with support for binary resources if you're willing to touch a bit of code. (I want to thank Ed Merks and Kenn Hussey for pointing me the way initially.) That's great, but now you have the problem of sticking the user with an unreadable binary format. Wouldn't it be cool to support both formats, so the user could take the same data and save it in the format most appropriate for his or her needs? It turns out that that's easy to do too, and it took me (much) less time to figure out then it took to write this blog post. And actually, I need to get on with it, because I'm working this weekend and I really don't want to be. So without even further ado, let's follow the heuristic I mentioned above.

First, we know that we need to support multiple resource types. How do we change that? Well, resources are setup in the foo.model plugin. In foo.util we can see the various factories. So we need to create two new pieces. (Actually, we don't have to do this, but it makes it a little easier to see what it happening.) So we could look at what the existing resource handlers are doing -- say FooResourceImpl -- and copy and modify it with the appropriate changes:

public class FooResourceImpl extends XMLResourceImpl {

    public FooResourceImpl(URI uri) {
        super(uri);
    }
}

Suggests that we probably want something like the below (although it's probably unnecesary):

public class FooBinaryResourceImpl extends BinaryResourceImpl {

    public FooBinaryResourceImpl(URI uri) {
        super(uri);
    }
}

I told you it was ridiculously simple. Similarly we want:

public class FooBinaryResourceFactoryImpl implements Resource.Factory {
    public Resource createResource(URI uri) {
        return new FooBinaryResourceImpl(uri);
    }
};

Now, we want to support two different kinds of files. We could do all sorts of funky stuff there, but let's keep it simple. We'll just use different file extensions. This part may seem just a bit mysterious, just because it involves a number of different artifact types. But first, we know that genmodel defines the file extensions. We can add one for the binary resources in the model section. Let's use "foo,foobin". We're only doing that for consistency because the real action is elsewhere. Since we're now dealing with editor stuff it makes sense to look there for what we want. And if we look at the FooModelWizard, we can see that there is a reference to _UI_FooEditorFilenameExtensions. That's the bit we want to change, and because genmodel won't write over .properties files, we'll need to edit it there:

[foo.editor/plugin.properties]
...
_UI_FooEditorFilenameExtensions = foo,foobin
...

We next need to tell the editor what to do with those binary files when it gets them. The natural place to do that is when we setup the editing domain for FooEditor, and we'll do it at the end of the method call:

/**
  * This sets up the editing domain for the model editor.  
  * 
  * @generated NOT
  */
 protected void initializeEditingDomain() {
...
  editingDomain.getResourceSet().getResourceFactoryRegistry().getExtensionToFactoryMap().put("foobin", new FooBinaryResourceFactoryImpl());
 }

OK, now the more challenging part. How to actually allow the user to select the file type? Again the obvious place to look is in new file wizard. We just need one change here, to tell the wizard what to do when the user selects the binary extension:

FooModelWizard
/**
* Do the work after everything is specified.
* 
* 
>>  * @generated NOT
*/
@Override
public boolean performFinish() {
...  
        // Create a resource set
        //
        ResourceSet resourceSet = new ResourceSetImpl();
>>resourceSet.getResourceFactoryRegistry().getExtensionToFactoryMap().put("foobin", new FooBinaryResourceFactoryImpl());
...
}

Now, for extra credit, how can we let the user change the file type once a file has been created? Well, we know the user picks the file type in the editor, so let's look there. And here we go..we'll show the entire method, but it is only the first 15 lines or so that have changed:

/**
*  
* 
* @generated NOT
*/
protected void doSaveAs(URI uri, IEditorInput editorInput) {
    ResourceSet resourceSet = editingDomain.getResourceSet();
    Resource resource = resourceSet.getResources().get(0);
    if (!uri.fileExtension().equals(resource.getURI().fileExtension())) {
        Resource oldResource = resource;
        if (uri.fileExtension().equals("foobin")) {
            resource = new FooBinaryResourceImpl(uri);
        } else if (uri.fileExtension().equals("foo")) {
            resource = new FooResourceImpl(uri);
        } else {
            throw new RuntimeException("Unexpected resource type: " + uri.fileExtension());
        }
        resource.getContents().addAll(oldResource.getContents());
        oldResource.getContents().clear();
        resourceSet.getResources().remove(oldResource);
        resourceSet.getResources().add(0, resource);
    }
    resource.setURI(uri);
    setInputWithNotify(editorInput);
    setPartName(editorInput.getName());
    IProgressMonitor progressMonitor = getActionBars().getStatusLineManager() != null ? getActionBars().getStatusLineManager().getProgressMonitor()
: new NullProgressMonitor();
    doSave(progressMonitor);
}

All we are doing there is creating the new resource, moving the contents over to it and replacing the old resource with the new one. (Don't be afraid to muck with resources and resource sets, they aren't as fragile as you may think.) That's it. I'm sure I missed something here that I'll discover later, but everything works.

Back to the original use case. I needed to support the xml format because I expect people to use the data in the application in a lot of different ways. But I really needed the binary format because my application produces very large files and those files take a long time to load. (When you load these files in the editor, the UI is frozen.) One file took:

19 m, 57 s

Using the binary format improved that a little bit:

00 m, 00 s, 419 milliseconds

Part of that is due to the fact that reference resolution can be deferred, but the entire file can be easily navigated without perceptible delay. The whole user experience is dramatically altered. That's a lot of functionality to gain with really some pretty small changes! And that's why taking the time to grasp more complex abstractions is worth the effort. What's interesting is that once you understand them, they don't seem at all complex!