{"id":740,"date":"2013-03-31T12:07:08","date_gmt":"2013-03-31T06:37:08","guid":{"rendered":"http:\/\/ramkulkarni.com\/blog\/?p=740"},"modified":"2013-03-31T12:07:08","modified_gmt":"2013-03-31T06:37:08","slug":"understanding-ast-created-by-mozilla-rhino-parser","status":"publish","type":"post","link":"http:\/\/ramkulkarni.com\/blog\/understanding-ast-created-by-mozilla-rhino-parser\/","title":{"rendered":"Understanding AST created by Mozilla Rhino parser"},"content":{"rendered":"

For an application I am developing, I needed to get all functions and variables declared in JavaScript code. Because the application I am developing is in Java, I started looking for readily\u00a0available\u00a0JavaScript parser written in Java. I found Mozilla Rhino<\/a> to be a good fit for my requirements and decided to use it.<\/p>\n

Parsing with Rhino is quite simple and well documented.<\/p>\n

\n
private<\/span> AstRoot parse<\/span>(<\/span>String src,<\/span> int<\/span> startLineNum)<\/span>\n\tthrows<\/span> IOException {<\/span>\n\n\tCompilerEnvirons env =<\/span> new<\/span> CompilerEnvirons();<\/span>\n\tenv.<\/span>setRecoverFromErrors<\/span>(<\/span>true<\/span>);<\/span>\n\tenv.<\/span>setGenerateDebugInfo<\/span>(<\/span>true<\/span>);<\/span>\n\tenv.<\/span>setRecordingComments<\/span>(<\/span>true<\/span>);<\/span>\n\n\tStringReader strReader =<\/span> new<\/span> StringReader(<\/span>src);<\/span>\n\n\tIRFactory factory =<\/span> new<\/span> IRFactory(<\/span>env);<\/span>\n\treturn<\/span> factory.<\/span>parse<\/span>(<\/span>strReader,<\/span> null<\/span>,<\/span> startLineNum);<\/span>\n\n}<\/span><\/pre>\n<\/div>\n

\nAstNode class has methods like getSymbols and getSymbolTable which I presumed would give me all variables and functions declared in the code. However these methods returned only functions and not variables declared. So I had to traverse the tree and get all symbols. I thought I would use visit method of AstRoot to process all nodes.<\/p>\n
\n
root.<\/span>visit<\/span>(<\/span>new<\/span> NodeVisitor()<\/span> {<\/span>\n\n\t@Override<\/span>\n\tpublic<\/span> boolean<\/span> visit<\/span>(<\/span>AstNode node)<\/span> {<\/span>\n\t\tint<\/span> nodeType  =<\/span> node.<\/span>getType<\/span>();<\/span>\n\t\t\/\/TODO: process the node based on node type<\/span>\n\t\treturn<\/span> true<\/span>;<\/span> \/\/process children<\/span>\n\t}<\/span>\n});<\/span><\/pre>\n<\/div>\n
However the above code threw ClassCastException exception in the visit method of ScriptNode class.<\/p>\n
java.lang.ClassCastException<\/span>: org.mozilla.javascript.Node cannot be cast to org.mozilla.javascript.ast.AstNode \u00a0 \u00a0at org.mozilla.javascript.ast.ScriptNode.visit(ScriptNode.java:312<\/span>)<\/p>\n
And the test code was a simple JS script –<\/p>\n
\n
function<\/span> test()\n{\n\tvar<\/span> k =<\/span> 10<\/span>;\n}\n\ni =<\/span> 100<\/span>;<\/pre>\n<\/div>\n
So finally I decided to traverse the AST from root and process each node. That is when I realized Rhino created AST differently from that AST I had worked with earlier. The AST I had worked with created nodes with their children as array of nodes. If you start with the root and visit each child, you are sure to traverse the entire tree and hence the code.<\/p>\n
However Rhino does not create nodes with array of child nodes. Each node has methods to access the first node and then you can traverse\u00a0remaining\u00a0child nodes by accessing ‘next’ member. So instead of an array, it creates linked list of child nodes, which is not such a big problem. However a few things\u00a0surprised\u00a0me when I traversed the tree.<\/p>\n
I started with the root node and for each node I followed following rules –<\/p>\n
    \n
  1. Get first child and traverse the linked list by calling next till the next is null<\/li>\n
  2. Call next on the parent node and repeat the loop.<\/li>\n<\/ol>\n
    Here is what I found –<\/p>\n
      \n
    1. When you traverse as above, you will only visit function name, and not function body<\/li>\n
    2. If you want to visit function body, you will have to call getFunctions (or getFunctionNode) method of ScriptNode. Both AstNode and FunctionNode are of type ScriptNode.<\/li>\n
    3. If you traverse the node as I described above (children first and then next node), you are not\u00a0guaranteed\u00a0to visit all AST nodes.<\/li>\n<\/ol>\n
      For the simple snippet of JS code above, the AST creates is as follows<\/p>\n
      \"Rhino<\/a><\/p>\n
      As you can see, only function Name node is in the tree. AstRoot has two children – Name (Function) and Node (Expr_Result). Expr_Result node has one child – Node (setName). SetName has two children- Name (BindName) and NumberLiteral. This tree represents the entire code, except body of the function test. As mentioned earlier, you can get body of the function by calling getFunctionNode on AstRoot.<\/p>\n
      Now, if you inspect Name (BindName) node, you will see that it’s parent is Assignment node. Assignment has left node as Name(BindName) and right node as NumberLiteral. However\u00a0Assignment\u00a0is not part of the tree if you traverse it as described above.<\/p>\n
      So you also need to consider type \u00a0(type constants are decalred in\u00a0org.mozilla.javascript.Token class) and parent of the node when processing the AST created by Rhino.<\/p>\n
      -Ram Kulkarni<\/p>\n
       <\/p>\n","protected":false},"excerpt":{"rendered":"
      For an application I am developing, I needed to get all functions and variables declared in JavaScript code. Because the application I am developing is in Java, I started looking for readily\u00a0available\u00a0JavaScript parser written in Java. I found Mozilla Rhino to be a good fit for my requirements and decided to use it. Parsing with … <\/p>\n
      Continue reading “Understanding AST created by Mozilla Rhino parser”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"Understanding AST created by Mozilla Rhino parser http:\/\/wp.me\/p2g9O8-bW","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[20,1],"tags":[67],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2g9O8-bW","jetpack-related-posts":[],"_links":{"self":[{"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/posts\/740"}],"collection":[{"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/comments?post=740"}],"version-history":[{"count":0,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/posts\/740\/revisions"}],"wp:attachment":[{"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/media?parent=740"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/categories?post=740"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/tags?post=740"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}