My tiny adventure of project generation
On this story I will explain
- Why I decided to create a project generator application
- The options I scrolled through
- How Kotlin helped me
- How I deployed my application to Google Cloud Platform
- How I took advantage of server-less
The Problem
I’ve been working (maybe for too long!) on my pet project which includes DSLs using Jetbrains MPS, and I will definitely get into much more details of this tool and how you can take advantage of its infinite possibilities in the future!
But today, I want to talk about one of the recent obstacles I faced which was the lack of easy support for building without an IDE in Jetbrains MPS. Sure, you can use the IDE to build a language, create the language plugin, even build an IDE containing your languages, but if you try to automate that on your Jenkins, you’ll find out it’s not as easy! MPS provides a custom build language which will generate ant files, but with the same reasoning you shouldn’t check-in your .class files into source control, the generated ant files or in fact any of the generated files (usually .java or .class files) should not be checked into source control too.
With a little search over the Jetbrains’ GitHub account, I found this IDE project which I think is for embedded software and is written using MPS. This was the breakthrough I was waiting for since my reported issue did not get me anywhere. This project uses Gradle to solve the issue I mentioned by calling a build solution to generate all the build scripts belonging to the project. There’s a chicken and egg situation here unfortunately because the build solution itself needs building also! But that’s solved by generating one build ant file and putting that on the project root. Having that, now you can use Gradle to build the build solution and from there, you can build any of the scripts that you might have on the project using Gradle. Their project for example has a build solution for each solution or language, there’s a build model for building the IDE and one for the distribution of the IDE.
After struggling for a couple of days I applied their solution into a sample project I had and I had a nice working solution. So far so good, but that just means for every language I create, I’ll have to go through a number of repetitive steps. As a developer that’s of course not ideal for me, and besides, I want to promote MPS languages soon, So I needed a good solution to overcome this problem… Enters “Project Generation”!
My Options
A lot of people are on guard when they hear “generating” something as they should be! Generating code or project comes with complications and costs of its own also. For example, “re-generating” is usually a very valid requirement that is not so easy to handle. Additionally, you have a new piece of code (the generator) that needs maintenance which if the generator is not used that often, would be a waste of time. You’ll end up updating the generators every month, but the actual generating happens once or twice a year only!
Anyways, let’s assume we have made the right choice and we do need to have a “project generator”. What options do we have? Being as old as I am, the first thing that crossed my mind was maven archetypes, but this solution didn’t survive long in the “options pool” because:
- I actually needed a Gradle project!
- If I want to make this accessible for everyone, I’d have to publish it on a public maven repository (TBH, I recently found out GitHub, GitLab, GCP and almost all cloud providers now provide such repositories where you can publish your maven artifacts for free or almost free)
- Personal experience showed not much flexibility when it comes to complicated generation logic
- I was more excited about other options 😅
Having a good experience with JHipster, I knew there’s a nice node.js generator they use which:
- Has interactive generator
- Is probably more targeted to tackle the problem I was trying to solve
- Had a cool name 😁
However:
- It would probably still need “workarounds” to work with Gradle
- Doesn’t have IDE support (maven archetypes are usually supported by IDEs)
- Was in a very unknown territory for me not having much experience with node.js
While I was reading about the generator that is used by JHipster (called yo) and how to customize it, a new option crossed my mind… Spring Initializr. This is a service I have been using for a very long time. It’s provided by Spring and is quite famous between us java developers. It’s open source and also it had a nice support by Intellij. As I started looking around the project, I found out it consists of two parts, the UI which is a React SPA and the backend service which is [drum roll please] a Spring application 😎
Implementing the generator
For me, just backend was enough so I abandoned the UI and just forked the backend project. I added a new module to the project, and started coding. TBH, initially I switched to Kotlin with the main reason of getting rid of the build errors I was getting because of my java files not having the copyright header and the order of imports not being satisfactory to his highness the spring-javaformat-maven-plugin! But as time went on, I was more and more happy with the decision because:
- A scripting language is more suitable for this kind of logic and Kotlin supports scripting
- Using Kotlin, I could “enhance” the existing code without changing the main code. This was very valuable because this is a live and active project on GitHub and I wanted to be able to get their latest updates in the future, so the less I had to change their code, the less possibility of conflicts I’d face in the future.
- This was a good opportunity for me to put my recent learnings of Kotlin into action!
So with that said, I branched out and added my own version of generator where you could add a “MPS Language” or a “MPS Model” which would then generate a Gradle-Kotlin project for you containing a language or a model with the Gradle setup as I explained above.
Running the generator “Service”
Next stop was running this. Of course it was running successfully locally, but how can I run it “on the cloud”? I was trying out Google Cloud Platform, so pretty much the first option I saw there was the App Engine. It was easy to setup, and easier to use! Nothing too complicated, but there was still a problem… I was being charged daily for a service which I’m the only user right now and might use 2–3 times a week and probably never in the future!
That’s when I started digging around “Server-less”. I knew, one of the goals of Spring Cloud Function project was to be able to run my project as server-less on major cloud providers, so I tried out their setup but unfortunately, it wasn’t successful because I think their GCP integration is in very early stages and was not easy for me to run. But more importantly, this would break one of my main rules not to make big changes to the main code. The main code was set up as a normal spring boot web application, and switching to server-less of course would require to change the whole structure of the application. That’s when I found this special GCP service which would help me with my situation: Cloud Run. Cloud Run allows you to run a docker image (which provides normal web interface) as server-less. This means I just needed to create a docker image from my spring boot web app, and Cloud Run would run it as server-less and practically charges me nothing since my service is not being used right now. Of course, not being “native server-less”, this would not be an optimal solution as it’s starting a rather big docker image when the first request arrives, but for me that was a very much acceptable sacrifice. Usually my first request would take seconds to get a response (as you can see on the demo gif), but the consecutive requests were pretty fast:
This was more than enough for me, but an option on top of my GCP console caught my eye “Set up Continuous Deployment”. I promised myself I wouldn’t spend more than a few hours to make it work and fortunately I didn’t have to! This part is using another GCP service called Google Cloud Build. You introduce your source code repository (there’s built-in support for GitHub), a trigger (e.x. push on master) and a pipeline yaml file (very similar to any other pipeline nowadays) specifying how your project is built and you’re good to go! Now I have a GitHub repository where I can make changes to my service and it will be automatically deployed to my GCP. Thanks developers all around the world for making this much progress on software development!
