Connecting with Modelix MPS plugin to a model repository through a webserver that has a custom SSL certificate

If you try to connect using the Cloud tool window of Modelix to a model repository that is served through a web server that has a custom SSL certifcate (something that would be typical to a company-internal webserver), then (at least at the moment of this writing), you will get an exception from the Modelix MPS plugin, saying something along the lines of:

ERROR - org.modelix.model.mpsplugin.CloudRepository - Failed to connected to https://address.to.your.modelrepo/model/
java.lang.RuntimeException: Unable to get the clientId by querying https://address.to.your.modelrepo/model/counter/clientId
at org.modelix.model.client.RestWebModelClient.getClientId(RestWebModelClient.kt:104)
at org.modelix.model.client.RestWebModelClient.(RestWebModelClient.kt:400)
at org.modelix.model.mpsplugin.CloudRepository$1.run(CloudRepository.java:80)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.ws.rs.ProcessingException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
8<--------------------------snip-------------------------------------------------

The way to solve this is:

  1. Get InstallCert.java from https://github.com/escline/InstallCert
  2. Run InstallCert.java, with your hostname and https port (you can either do javac InstallCert.java and then java InstallCert, or since java 11, you can do java --source 11 InstallCert.java directly), and press 1 when asked for input. This will add your localhost as a trusted keystore and generate a file called jssecacerts. It will throw an exception because your certificate is not yet in the store.
  3. Run InstallCert.java again to verify that the certificate is now added to the store: the connection should be OK and there should be no exceptions.
  4. Copy the generated jssecacerts file to your $MPS_HOME/jbr/lib/security folder.

Credits for the fix go to this blog post by mkyong: https://www.mkyong.com/webservices/jax-ws/suncertpathbuilderexception-unable-to-find-valid-certification-path-to-requested-target/.

If you are still here, the following is an explanation of why you had to do what you had to do: the issue is that the REST communication API used in Modelix (JAX-RS) doesn’t use the standard /lib/security/cacert store, but rather the /lib/security/jssacert store to make custom certificates work when connecting Modelix MPS client plugin with a Modelix model repository that is served on a web-server with a custom SSL certificate. Therefore, the standard way of importing into cacerts will not work.

Itemis artifacts for MPS 2019.3 through MPS 2020.1

A brief update on how the artifacts of mbeddr, mbeddr.platform, and IETS3 have been changed (for the people that have to deal with older MPS models made using one of the mentioned extension packages): up until MPS 2019.2, it was possible to download all artifacts from build.mbeddr.com. Today, I was trying to do this as well, but it seems all the artifacts have been wiped for mentioned MPS versions (maybe some space needed to be made free on build.mbeddr.com, or maybe some internal build process changed). Now you have to actually get the label of a version that has successfully built from build.mbeddr.com (e.g. 2019.3.22698.f4cbbf7 of the mbeddr platform or 2020.1.4698.e240d62 of IETS3.opensource, the latter leading to https://projects.itemis.de/nexus/content/repositories/mbeddr/org/iets3/opensource/2020.1.4698.e240d62/opensource-2020.1.4698.e240d62.zip) and then get the corresponding artifacts from https://projects.itemis.de/nexus/content/repositories/mbeddr. Especially for IETS3 (KernelF), this is really needed.

Note: since MPS 2020.2, it is possible to simply get release artifacts from github (e.g. https://github.com/IETS3/iets3.opensource/releases/download/nightly-2020.2.4726.e49ca16/org.iets3.opensource-with-dependencies-2020.2.4726.e49ca16.zip). And these artifacts even include the dependencies (such as mbeddr platform, which is needed by KernelF/IETS3).

MPS Extensions available as plugins on JetBrains marketplace

Let me tell you about something awesome that I have been working on with Mike Vlassiev from the JetBrains people (or rather, they have been working on it and I have done testing and provided feedback): the MPS Extensions (https://github.com/JetBrains/MPS-extensions) are now automatically made available on the JetBrains marketplace on every new MPS and MPS Extensions release.

This means that from now on all the MPS (https://www.jetbrains.com/mps/) plugins provided by MPS Extensions (https://jetbrains.github.io/MPS-extensions/extensions/plaintext-gen/) can be simply installed through File –> Settings –> Plugins –> Marketplace.

Why is that cool? Well, the MPS Extensions, which are maintened by the MPS Community and curated by Itemis (https://www.itemis.com/de/), are very useful plugins which make language developers even more productive than an out-of-the-box installed MPS can do.

Some of the most prominent examples in the MPS Extensions are:

  • Diagrammatic, tabular, and math notation for MPS languages (de.itemis.mps.editor.diagram, de.slisson.mps.tables, and de.itemis.mps.editor.math)
  • Incrementally maintained non-editable models from existing models based on model-to-model transformations (de.q60.mps.shadowmodels)
  • Template-based text generation (com.dslfoundry.plaintextgen)
  • … and many more at https://github.com/JetBrains/MPS-extensions!

And now all these goodies have just been made much more accessible to novice users of MPS via the JetBrains marketplace. This also means, that dependencies for plugins are automatically resolved, which is important for MPS Extensions, since the plugins there are built with a high level of re-usability in mind (and hence dependencies over plugins exist).

I think this is a very important step in the direction of getting the results of the community accessible to new users. A next good step will be to make all the MPS Extensions directly available via one meta-package.

Plaintextgen tutorial

The com.dslfoundry.plaintextgen extension (https://jetbrains.github.io/MPS-extensions/extensions/plaintext-gen/) for JetBrains MPS (https://www.jetbrains.com/mps/) enables a language designer to write model-to-text generation templates in pretty much the same way one writes model-to-model transformations in MPS. This is very useful for many reasons, the most prominent two of which are in my opinion: you don’t have or don’t want to build a generation target language for your text generation needs (e.g. because you need to make some model-to-text transformation very quickly) or if you want to work more like classical template-based text generation similar to Xtext.

This tutorial describes some of the most common uses of the plaintextgen.

Assumptions

This tutorial assumes that you have basic knowledge of MPS, i.e. how to create a new project and how to make a language with an editor and a generator.

How to use plaintextgen?

Before you can use plaintextgen, you need to install it first. You can either do so by downloading the MPS Extensions (https://github.com/JetBrains/MPS-extensions/releases) and copying them into your <MPS_Installation_Folder>\plugins folder, adding them to your global libraries (File –> Settings –> Build, Execution, Deployment –> Global Libraries), or by installing the com.dslfoundry.plaintextgen plugin from the MPS Marketplace (File –> Settings –> Plugins –> Marketplace).

Once you have com.dslfoundry.plaintextgen installed, you can use it within a generator template. The idea behind plaintextgen is to reproduce the look-and-feel of the textual editor aspect as much as possible, since both text generator and textual editor in principle handle the rendering of concrete syntax for a textual notation on your model. Therefore, I am not going to explain each detail and you can try to use the basis of your mental model of textual editors (e.g. vertical and horizontal collections, newlines, surrounding blocks with collections) on the plaintextgen language. Names are a bit different, e.g. vertical collection (from editor) versus VerticalLines, horizontal collection versus Line, CellModel versus Word, etc. You can see the metamodel of plaintextgen in the structure of the language.

If you feel that certain things are very different from the editors aspect, then this is most probably a bug or not-yet-implemented feature. Feel free to file an issue (or even better: make a pull request) on https://github.com/JetBrains/MPS-extensions for the plaintextgen component.

I will illustrate most of plaintextgen’s basic functionality by extending the generator of the example language com.dslfoundry.plaintextgen.example.testlang. What I will do is add a generator for a C++ file, based on an existing C++ listing from the ThinkC++ book by Allen B. Downey ( http://greenteapress.com/thinkcpp/index.html ). The listing in question is one of the accompanying code samples on github: https://github.com/AllenDowney/ThinkCPP/blob/master/code/cha11/cha11.cpp . The templating of the code is only to illustrate the mechanism and will probably be completely nonsensical, so don’t try to think too hard on what the end result would mean or whether it can be compiled 🙂 Now, if you have any suggestions for improvements, I’ll be happy to hear that in the comments.

If you want to reproduce this tutorial based on com.dslfoundry.plaintextgen.example.testlang, you can clone the https://github.com/JetBrains/MPS-extensions repository and open the MPS project under the code directory. The MPS version that is supported on master can be found at the key ext.mpsMajor in the file build.gradle in the root of the repository. If you need the code for an older version, there are maintenance/mps<version> branches. States of the branches are described at https://jetbrains.github.io/MPS-extensions/#current-versions. One last thing to add before we begin: I will add MPS links that you can click from here if you have the code project of the MPS Extensions repository open and it will open the specific node that the link points to.

So let’s get going!

Just for reference, com.dslfoundry.plaintextgen.example.testlang can be found here in the project:

First we will have a brief look at the test model (instance of the TestConcept concept) in com.dslfoundry.plaintextgen.testlang.sandbox:

So we have a TestConcept instance which has a property called testProperty with value TestValue. Then, it also has two children in its list of testSubConcepts with (respectively) testProperty values TestValue2 and TestValue3.

Let’s make our first change to the language by adding a new plaintextgen-based root template to its generator:

So far nothing special, it’s just the creation of a root template. The only thing you need to know is to import the com.dslfoundry.plaintextgen language (Ctrl+L+L and search for the language, then select it and press enter or double click it).

Now, we will import the text from https://github.com/AllenDowney/ThinkCPP/blob/master/code/cha11/cha11.cpp into our newly created root template:

So we only need to select the plain text from the C++ source file and then do execute the Paste Unstructured Text on the plaintextgen root template. Plaintextgen takes care of analyzing indentations and makes a full plaintextgen template for you.

Next, let’s feed the testProperty of the root TestConcept into the template:

This works again pretty much the same as the normal MPS model-to-model generator templating mechanism. We just add a property macro on the filename based on the testProperty value of the root TestConcept and then we add a filename extension .cpp. Then we run a build and perform a Preview Generated Text action (via right-click) on the model root. The result is a text file called TestValue.cpp which lists the hard-coded contents that we have imported in the last step.

To continue, we will feed the value of testProperty of the root TestConcept to the struct name:

Here we use the Split-extract Selection into Word intention of plaintextgen in order to make it a separate word that can be templated. Then we put a node macro for testProperty on the split-extracted word. When we build this and then again run a Preview Generated Text action, we see that the name Time has been replaced by TestValue, as would be expected.

As a next step, we will feed the testProperty values of the testSubConcepts children ( TestValue2 and TestValue3 in the model) to the struct members:

To do this, first we delete all but one of the struct members from the template. Now, we apply a LOOP macro on the template (again, just normal MPS generator macro stuff); N.B.: don’t forget to include the newline, or else you will miss newlines in the output. We finish this step off by doing a split-extract on the name of the member and feeding the testProperty value to it via a property macro. When we build this, the output is as expected: we have two members called TestValue2 and TestValue3.

Finally, we are going to add an implementation for each member based on the testProperty names of the testSubConcepts children:

Again, we delete all implementation functions except for one. Since we cannot apply a macro on multiple nodes, we need to wrap the multiple lines of the implementation function into one node. To do this, we use the Surround with Vertical Collection selection intention (N.B.: we did not press the light bulb, but pressed Alt+Enter to trigger the intention since it’s a special intention which can be executed on a selection of multiple nodes, provided by the language de.itemis.mps.selection.intentions). Now that the multiple lines are wrapped, we can apply a LOOP macro, do again the split-extract on the name of the implementation, and use a property macro to feed the value of the testProperty of each of the testSubConcepts children.

Of course, we could continue and really make this a much more refined example, but this should be enough for now to illustrate the major mode of working with plaintextgen to do your own templates.

A last remark: this tutorial took a template imported from plain text (a C++ file) as a basis and did some plaintextgen operations on it to get to the desired results. Of course, you can also build your own plaintextgen templates from scratch (adding new word, vertical lines, lines, etc.), but if you have a textual source, then I think the above method is much easier.

How to add a sandbox to an MPS language that doesn’t have one

Often times, when creating a new language, I skip the “Create Sandbox Solution” checkbox. Recently, I have learned that the sandboxes are actually not just solutions with the name sandbox. Because of their property of actually residing in the language folder itself, many MPS Extensions are actually using sandboxes to package example models with a language to show its use.

Now imagine that you have created a language without a sandbox, then you may wonder how to add one. Although the answer is very simple, I still describe it here, since it may not be immediately obvious.

I’ll following the com.jetbrains.etc naming convention in this example. Imagine you have a language called com.dslfoundry.mylanguage and which doesn’t have a sandbox yet. To add a sandbox to it:

  • Create a new solution called com.dslfoundry.mylanguage.sandbox
  • Add a new model to it called com.dslfoundry.mylanguage.sandbox (so normally, you just do an “add model” and then remove the dot at the end of com.dslfoundry.mylanguage.sandbox.)
  • Open a file explorer and navigate to the folder of your language (usually that is <project_folder>/languages/com.dslfoundry.mylanguage)
  • Move the entire folder of the earlier created com.dslfoundry.sandbox solution to your language folder
  • Now rename com.dslfoundry.sandbox to just sandbox
  • To make the sandbox show up on your project, you have to right-click your project in the logical view and select properties, then click the + to add the .msd-file of the sandbox solution.

Handy non-default settings for MPS

MPS comes with pretty good default settings, but there are some settings that I really like to change as soon as I have a fresh MPS IDE in front of my nose. Below is a description of such settings.

Hide empty aspect tabs

By default, MPS always shows all tabs for all aspects (also aspects that are empty) for each concept. I have found that I keep more overview when the empty aspects of a concept are hidden. You can change this, by setting the Aspect Tabs option to Each aspect node in a separate tab:

Hide generator warnings that you cannot fix

So far (right up until MPS 2019.1), MPS shows some really annoying “Failed to replace dynamic reference” generator warnings. They are annoying because they pollute the build logging and you cannot do anything about them as a language engineer. You can remove these (without being afraid that your languages will not be checked when you commit, because MPS checks all modified languages and models on every commit) by unchecking Show warnings in the generator settings:

Disable language checking in every build

When working incrementally on a language implementation, I often run many rebuilds with every small delta that I want to test for my language. Since model/language checking is not incremental, having a check every time when I need to build a language can take a lot of cumulatively collected overhead time. Therefore, I like to, by default, turn off the model checking for every build.

There is a caveat, though: since MPS’ language development workflow currently relies on languages/models being checked in the IDE and not in command line/buildserver build, you should do model checking before checking in! I still change the default setting, which requires more discipline from me, but saves me some time.

Disabling continous modelchecking can be accomplished by unchecking the item Build –> Check models before generation item in the menu:

Prevent auto-opening of last-opened project

For me, the reason of shutting down or restarting MPS is usually because I need to switch between projects or because I need to play with certain settings. This makes the default feature of reloading your last opened project upon MPS startup not very useful. I turn it off here:

Memory Indicator

This piece of information is provided thanks to Hristina Moneva (see coments).

The memory indicator does not only give you a way to see how much memory is allocated to the VM that MPS and your memory use, but it also gives you a way to do garbage collection manually. It is located in the lower right corner and looks like this:

Adding the memory indicator can also be done via the settings menu:

First prototype of plaintextflow released

The mps-plaintextgen plugin has been upgraded with new functionality for fine-grained control of whitespaces. Such functionality is very useful for languages that are fiddly with whitespace such as reStructuredText or where you just want to have very fine control (such as in JSON or in some instances Python).

Plaintextflow is part of the plaintextgen plugin, but can be imported as a separate language, so existing functionality of plaintextgen is not disturbed.

The plugin can be downloaded from https://github.com/DSLFoundry/mps-plaintextgen/releases/download/nightly-0022/com.dslfoundry.plaintextgen-MPS-2018.2-RC2.zip and will soon be available from the JetBrains MPS plugin repository (https://plugins.jetbrains.com/mps).

MPS Logging

For many purposes of end-users and language engineers , the standard MPS mechanism with tooltips and Event Log is sufficient:

However, in some cases more information is needed. This is where the MPS log files come into play. There are two ways to reach this log:

  • You can use Help –> Show Log in Explorer (or the Linux/Mac equivalent of this) from the MPS main menu. This will point you to the rotating idea.log (usually located in your $HOME/.MPS<version>/system/log directory). In this case you will either inspect the log using your favorite text editor, or you can use a dedicated log4j-compatible log viewer (just google for it).
  • You can look back at the command line from where you started MPS. In case of Linux (and I think Mac), this will go automatically, if you start MPS from a console using mps.sh (which is located in the bin directory of your MPS installation), but in case of Windows, you have to adapt your mps.bat (which is also located in the bin directory of your MPS installation) as follows: replace all the strings that say “javaw” with “java” (currently there are two strings; this will ensure that there is a console open next to your MPS instance into which you can look in real-time).