Challenges in implementing ColdFusion Builder

ColdFusion 10 and ColdFusion Builder 2.0.1 released this week, on 14th May. My article on What’s new in ColdFusion Builder 2.0.1 is published on the Adobe Developer Connection. I have been working on ColdFusion Builder for close to 3.5 years now and the release of updater this week has put me in a mood to reflect on challenges we have faced so far.

The decision to create a new IDE for ColdFusion was taken by the team during the research phase of ColdFusion 9. When we visited ColdFusion customers and asked which features they would like in the next release of ColdFusion, the IDE came up at the top of the list, even though it is not a server feature per se. There were already three IDEs for CFML at that time; Dreamweaver, Homesite+ and CFEclipse. Each one has its own strengths. Users of each of these IDEs expected all the features of their favourite IDE, and some additional features, in ColdFusion Builder (CFB). So selection of features for CFB has always been a tough job in every release. What made the implementation tougher in CFB 1.0 was 1. limited resources (the same team working on server and IDE) 2. minimal or lack of Eclipse development experience in the team 3.window of 1-1.5 years to implement it (so that we could release it close to the release of CF9).

The decision to base the new IDE on Eclipse was easy. Adobe already had Flash Builder which was developed on the Eclipse platform. And the team had mostly Java programming expertise, which is the programming language for developing Eclipse plugins. Also the extensible architecture of Eclipse meant that we could use existing plugins for features like source code control, build etc. The next thing to decide was what existing work we could leverage – we had options of CFEclipse and Aptana. CFML file can contain HTML, CSS, JavaScript, along with CFML code. So we had to build an IDE that supported editing of all these files. At that time Aptana had very good editors for HTML, CSS, JavaScript and XML. So we decided to build CFB on top of Aptana. We did use some of the features from CFEclipse too, like CFML language dictionary and code snippet.

There have been many challenges in developing ColdFusion Builder – some technical, others non-technical. In this post I will only write about some of the technical challenges.

Code Partitioning:

Integrating CFML editor in Aptana proved to be quite a difficult job. Aptana had created a framework where editors for other languages could be embedded into HTML editor. Most languages that Aptana had already implemented with this framework had clear separation points from HTML e.g. JavaScript code is in <script> tag, CSS is in <style> tag, Ruby code is within <%%>. Therefore partitioning code for different languages was easy for them. However CFML tags look and behave like HTML and can intermix with HTML and JavaScript. So partitioning CFML code using Aptana’s framework was difficult. Code has to be partitioned correctly in Eclipse editors because Eclipse associates one editor with each partition and delegates editor tasks like code colorization, code assist, code folding etc. to that editor. So it was very important that we got this right. We got around this issue by asking Aptana to make some changes in their code and also implementing number of work-around in our code.

Parser:

We decided to use the CFML parser that we had in server for our new IDE. However it required few changes, the primary one being implementing error recovery. When the server executes code and encounters error, it immediately stops execution and reports error. However IDE needs to mark the error, recover till the next logical point in the code and continue parsing. I do not claim that we have implemented 100% error recovery in CFB, but we have tried to do it to a large extent.

When you are typing in the editor, the previously parsed state of the file changes. So all the data that you had collected after parsing, to provide code assist, code folding and other editor features, become obsolete. But you cannot parse the file after every keystroke; that would make the editor very slow. So we parse the document when there is a short pause in typing.

We also attempted to parse code incrementally i.e. localize parsing to the area where code is changing. There are many things to consider when implementing incremental parser- you need to identify logical block around the code that the user is typing and only parse that part, create AST (Abstract Syntax Tree) and replace corresponding sub tree in the main AST. You also need to consider impact of the code that is being edited on the other parts of the code. All this makes implementing incremental parsing extremely complicated. We did try to implement this, but could not finish it because of lack of time.

But to improve responsiveness of the editor we implemented few other techniques, such as stopping the parser if file is changed because of typing (and re-parsing it when there is a pause in typing), keeping main UI thread of the editor as free as possible by doing only UI changes in that thread and offloading remaining work to worker threads etc.

Synchronous Framework:

Aptan’s framework for processing colorization, code folding and outline is highly synchronous and code is tokenized for supporting these features after every key stroke. Ssince this happens in the main UI thread of Eclipse, the IDE becomes unresponsive till the process is finished. The problem is not so much visible in small files, but when editing large files, the lag could be significant. We have tried to make tokenization process asynchronous in CFML editor (and it is not done after every key stroke). But there is still scope for improvement in handling code colorization.

Performance Testing:

CFB tries aggressively to provide code assist; for buit-in tags/functions, user defined tags/functions, CFCs, methods in CFCs, datasources, SQL code assist in CFQuery, ORM code assist, code assist for included files etc. However this has caused some performance issues as well, particularly providing code assist for CFCs and ORM entities. We have fixed many such performance issues in the last two releases. Finding such bugs in our testing is more of a challenge than actually fixing them. There have been instances when we had run our tests for performance, found no issues, but users reported performance bugs which they found because of their specific set up.

Development of an IDE for a language is very challenging – it has aspects of developing parser, language runtime and user interface all in one application. It has been very satisfying experience for me and I am very proud of what our small team has managed to achieve in ColdFusion Builder. Yes, there is still a lot of scope for improvement. In hindsight we could have done a few things differently. But in the given circumstances we have tried to do our best.

-Ram Kulkarni