ColdFusion 10 and ColdFusion Builder 2.0.1 released this week, on 14th May. My article on What’s new in ColdFusion Builder 2.0.1 is published on the Adobe Developer Connection. I have been working on ColdFusion Builder for close to 3.5 years now and the release of updater this week has put me in a mood to reflect on challenges we have faced so far.
The decision to create a new IDE for ColdFusion was taken by the team during the research phase of ColdFusion 9. When we visited ColdFusion customers and asked which features they would like in the next release of ColdFusion, the IDE came up at the top of the list, even though it is not a server feature per se. There were already three IDEs for CFML at that time; Dreamweaver, Homesite+ and CFEclipse. Each one has its own strengths. Users of each of these IDEs expected all the features of their favourite IDE, and some additional features, in ColdFusion Builder (CFB). So selection of features for CFB has always been a tough job in every release. What made the implementation tougher in CFB 1.0 was 1. limited resources (the same team working on server and IDE) 2. minimal or lack of Eclipse development experience in the team 3.window of 1-1.5 years to implement it (so that we could release it close to the release of CF9).
There have been many challenges in developing ColdFusion Builder – some technical, others non-technical. In this post I will only write about some of the technical challenges.
We decided to use the CFML parser that we had in server for our new IDE. However it required few changes, the primary one being implementing error recovery. When the server executes code and encounters error, it immediately stops execution and reports error. However IDE needs to mark the error, recover till the next logical point in the code and continue parsing. I do not claim that we have implemented 100% error recovery in CFB, but we have tried to do it to a large extent.
When you are typing in the editor, the previously parsed state of the file changes. So all the data that you had collected after parsing, to provide code assist, code folding and other editor features, become obsolete. But you cannot parse the file after every keystroke; that would make the editor very slow. So we parse the document when there is a short pause in typing.
We also attempted to parse code incrementally i.e. localize parsing to the area where code is changing. There are many things to consider when implementing incremental parser- you need to identify logical block around the code that the user is typing and only parse that part, create AST (Abstract Syntax Tree) and replace corresponding sub tree in the main AST. You also need to consider impact of the code that is being edited on the other parts of the code. All this makes implementing incremental parsing extremely complicated. We did try to implement this, but could not finish it because of lack of time.
But to improve responsiveness of the editor we implemented few other techniques, such as stopping the parser if file is changed because of typing (and re-parsing it when there is a pause in typing), keeping main UI thread of the editor as free as possible by doing only UI changes in that thread and offloading remaining work to worker threads etc.
Aptan’s framework for processing colorization, code folding and outline is highly synchronous and code is tokenized for supporting these features after every key stroke. Ssince this happens in the main UI thread of Eclipse, the IDE becomes unresponsive till the process is finished. The problem is not so much visible in small files, but when editing large files, the lag could be significant. We have tried to make tokenization process asynchronous in CFML editor (and it is not done after every key stroke). But there is still scope for improvement in handling code colorization.
CFB tries aggressively to provide code assist; for buit-in tags/functions, user defined tags/functions, CFCs, methods in CFCs, datasources, SQL code assist in CFQuery, ORM code assist, code assist for included files etc. However this has caused some performance issues as well, particularly providing code assist for CFCs and ORM entities. We have fixed many such performance issues in the last two releases. Finding such bugs in our testing is more of a challenge than actually fixing them. There have been instances when we had run our tests for performance, found no issues, but users reported performance bugs which they found because of their specific set up.
Development of an IDE for a language is very challenging – it has aspects of developing parser, language runtime and user interface all in one application. It has been very satisfying experience for me and I am very proud of what our small team has managed to achieve in ColdFusion Builder. Yes, there is still a lot of scope for improvement. In hindsight we could have done a few things differently. But in the given circumstances we have tried to do our best.