{"id":1210,"date":"2014-04-04T19:57:15","date_gmt":"2014-04-04T14:27:15","guid":{"rendered":"http:\/\/ramkulkarni.com\/blog\/?p=1210"},"modified":"2014-04-04T19:57:15","modified_gmt":"2014-04-04T14:27:15","slug":"parsing-javascript-code-using-mozilla-rhino","status":"publish","type":"post","link":"http:\/\/ramkulkarni.com\/blog\/parsing-javascript-code-using-mozilla-rhino\/","title":{"rendered":"Parsing JavaScript code using Mozilla Rhino"},"content":{"rendered":"

Last year I had blogged “Understanding AST created by Mozilla Rhino parser<\/a><\/strong>“, where I exaplained how to traverse AST to get all functions and variables. Since then I have received some comments, public and private, to provide sample code. So here is an example of how to parse JavaScript code and get all functions and variables using Mozilla Rhino.<\/p>\n

Note that my intention here is not to build the complete symbol table for any given JavaScript code. And I have not run a lot of test cases on this code. The idea is to show how AST could be traversed and how hierarchy of symbols (functions and variables) be built . <\/strong><\/p>\n

So, for example, if you feed following JavaScript code to this program –<\/p>\n

\n
function<\/span> func1()\n{\n\tvar<\/span> local1 =<\/span> 10<\/span>;\n\n\tglobal1 =<\/span> 20<\/span>;\n}\n\nobj1 =<\/span> {\n\tobjFunc1 :<\/span> function<\/span>()\n\t{\n\t\tvar<\/span> local2 =<\/span> 30<\/span>;\n\t},\n\tobjProp1 :<\/span> \"prop1\"<\/span>\n}<\/pre>\n<\/div>\n
You will get following output –<\/p>\n
\n
Function : func1\n\t\tlocal1\n\tglobal1\n\tobj1\n\tobj1\n\t\tobjFunc1\n\t\tFunction : objFunc1\n\t\t\tlocal2\n\t\tobjProp1<\/pre>\n<\/div>\n
Yes, I know there are duplicates for object and closure – because they are first processed as variable names and then object\/closure. This can be easily fixed, but I am going to leave it that way to keep the code simple.<\/p>\n
I am not going to produce the complete source code here, but will explain some important parts of the program.<\/p>\n
The program uses visitor pattern to first traverse AST and then to print symbol hierarchy. It contains following classes –<\/p>\n
JSSymbol<\/a> : This class holds reference to AST node and child elements. We will create hierarchy of objects of this class<\/p>\n
JSNodeVisitor<\/a>: implements Rhino’s NodeVisitor interface and builds hierarchy of JSSymbo<\/p>\n
IJSSymbolVisitor<\/a> : visitor interface for JSSymbol<\/p>\n
JSSymbolVisitor<\/a> : implements IJSSymbolVisitor and prints hierarchy of JSSymbol objects<\/p>\n
JSErrorReporter<\/a> : implements ErrorReporter interface of Rhino and prints syntax errors in the JS script<\/p>\n
RhinoDemo<\/a> : main program<\/p>\n
paseJS function in RhinoDemo<\/a> class parses the script and gets the AstRoot node. It then calls visit method of the root node, passing JSNodeVisitor<\/a> to it. After building the symbol heir achy, it prints it using JSSymbolVisitor<\/a>.<\/p>\n
\n
public<\/span> void<\/span> parseJS<\/span> (<\/span>String filePath)<\/span> throws<\/span> Exception\n{<\/span>\n\tCompilerEnvirons env =<\/span> new<\/span> CompilerEnvirons();<\/span>\n\n\tenv.<\/span>setRecoverFromErrors<\/span>(<\/span>true<\/span>);<\/span>\n\n\tFileReader strReader =<\/span> new<\/span> FileReader(<\/span>filePath);<\/span>\n\n\tIRFactory factory =<\/span> new<\/span> IRFactory(<\/span>env,<\/span> new<\/span> JSErrorReporter());<\/span>\n\tAstRoot rootNode =<\/span> factory.<\/span>parse<\/span>(<\/span>strReader,<\/span> null<\/span>,<\/span> 0);<\/span>\n\n\tJSNodeVisitor nodeVisitor =<\/span> new<\/span> JSNodeVisitor();<\/span>\n\n\trootNode.<\/span>visit<\/span>(<\/span>nodeVisitor);<\/span>\n\n\tnodeVisitor.<\/span>getRoot<\/span>().<\/span>visit<\/span>(<\/span>new<\/span> JSSymbolVisitor());<\/span>\n}<\/span><\/pre>\n<\/div>\n
visit method of JSNodeVisistor calls addToParent method which actually builds symbol hierarchy<\/p>\n
\n
private<\/span> void<\/span> addToParent<\/span>(<\/span>AstNode node)<\/span>\n{<\/span>\n\tif<\/span> (<\/span>root ==<\/span> null<\/span>)<\/span>\n\t{<\/span>\n\t\troot =<\/span> new<\/span> JSSymbol(<\/span>node);<\/span>\n\t\tfunctionsStack.<\/span>push<\/span>(<\/span>root);<\/span>\n\t\tcurrentFuncEndOffset =<\/span> node.<\/span>getAbsolutePosition<\/span>()<\/span> +<\/span> node.<\/span>getLength<\/span>();<\/span>\n\t\treturn<\/span>;<\/span>\n\t}<\/span>\n\n\tif<\/span> (<\/span>functionsStack.<\/span>size<\/span>()<\/span> ==<\/span> 0)<\/span>\n\t\treturn<\/span>;<\/span>\n\n\tint<\/span> nodeType =<\/span> node.<\/span>getType<\/span>();<\/span>\n\n\t\/\/we will track only variables and functions<\/span>\n\tif<\/span> (<\/span>nodeType !=<\/span> Token.<\/span>FUNCTION<\/span> &&<\/span> nodeType !=<\/span> Token.<\/span>VAR<\/span> &&<\/span> nodeType !=<\/span> Token.<\/span>OBJECTLIT<\/span> &&<\/span>\n\t\t\t!(<\/span>nodeType ==<\/span> Token.<\/span>NAME<\/span> &&<\/span> node.<\/span>getParent<\/span>()<\/span> instanceof<\/span> ObjectProperty))<\/span>\n\t{<\/span>\n\t\tif<\/span> (<\/span>isVariableName(<\/span>node))<\/span>\n\t\t{<\/span>\n\t\t\t\/\/check if it is in the current function<\/span>\n\t\t\tString symbolName =<\/span> ((<\/span>Name)<\/span>node).<\/span>getIdentifier<\/span>();<\/span>\n\t\t\tJSSymbol currentSymContainer =<\/span> functionsStack.<\/span>peek<\/span>();<\/span>\n\t\t\tif<\/span> (!<\/span>currentSymContainer.<\/span>childExist<\/span>(<\/span>symbolName))<\/span>\n\t\t\t{<\/span>\n\t\t\t\t\/\/this is a global symbol<\/span>\n\t\t\t\troot.<\/span>addChild<\/span>(<\/span>node);<\/span>\n\t\t\t}<\/span>\n\t\t}<\/span>\n\t\treturn<\/span>;<\/span>\n\t}<\/span>\n\n\tif<\/span> (<\/span>node.<\/span>getType<\/span>()<\/span> ==<\/span> Token.<\/span>VAR<\/span> &&<\/span> node instanceof<\/span> VariableInitializer ==<\/span> false<\/span>)<\/span>\n\t\treturn<\/span>;<\/span>\n\n\tJSSymbol currSym =<\/span> null<\/span>;<\/span>\n\n\tJSSymbol parent =<\/span> functionsStack.<\/span>peek<\/span>();<\/span>\n\tif<\/span> (<\/span>parent.<\/span>getNode<\/span>().<\/span>getAbsolutePosition<\/span>()<\/span> +<\/span> parent.<\/span>getNode<\/span>().<\/span>getLength<\/span>()<\/span> ><\/span> node.<\/span>getAbsolutePosition<\/span>())<\/span>\n\t{<\/span>\n\t\tcurrSym =<\/span> new<\/span> JSSymbol(<\/span>node);<\/span>\n\t\tparent.<\/span>addChild<\/span>(<\/span>currSym);<\/span>\n\t}<\/span>\n\telse<\/span> \/\/outside current function boundary<\/span>\n\t{<\/span>\n\t\t\/\/pop current parent<\/span>\n\t\tfunctionsStack.<\/span>pop<\/span>();<\/span>\n\t\taddToParent(<\/span>node);<\/span>\n\t\treturn<\/span>;<\/span>\n\t}<\/span>\n\n\t\/\/currSym is already set above<\/span>\n\tif<\/span> (<\/span>nodeType ==<\/span> Token.<\/span>FUNCTION<\/span> ||<\/span> nodeType ==<\/span> Token.<\/span>OBJECTLIT<\/span>)<\/span>\n\t{<\/span>\n\t\tAstNode parentNode =<\/span> node.<\/span>getParent<\/span>();<\/span>\n\t\tAstNode leftNode =<\/span> null<\/span>;<\/span>\n\t\tif<\/span> (<\/span>parentNode.<\/span>getType<\/span>()<\/span> ==<\/span> Token.<\/span>ASSIGN<\/span>)<\/span>\n\t\t{<\/span>\n\t\t\tleftNode =<\/span> ((<\/span>Assignment)<\/span>parentNode).<\/span>getLeft<\/span>();<\/span>\n\t\t}<\/span>\n\t\telse<\/span> if<\/span> (<\/span>parentNode instanceof<\/span> ObjectProperty)<\/span>\n\t\t{<\/span>\n\t\t\tleftNode =<\/span> ((<\/span>ObjectProperty)<\/span>parentNode).<\/span>getLeft<\/span>();<\/span>\n\t\t}<\/span>\n\n\t\tif<\/span> (<\/span>leftNode instanceof<\/span> Name)<\/span>\n\t\t\tcurrSym.<\/span>setName<\/span>(((<\/span>Name)<\/span>leftNode).<\/span>getIdentifier<\/span>());<\/span>\n\n\t\tfunctionsStack.<\/span>push<\/span>(<\/span>currSym);<\/span>\n\t\tcurrentFuncEndOffset =<\/span> node.<\/span>getAbsolutePosition<\/span>()<\/span> +<\/span> node.<\/span>getLength<\/span>();<\/span>\n\t}<\/span>\n}<\/span><\/pre>\n<\/div>\n
The above function uses Stack to keep track of current parent symbol (function or Object). It checks if the node being processed is child of the symbol at the top of functionsStack<\/span> by comparing node’s absolute position (AstNode.getAbsolutePosition) with end offset of the current (top) symbol. Since AST is visited in depth first manner, this logic works fine.<\/p>\n
Remaining code in this program is quite simple. Download Eclipse project for this program<\/a> if you use Eclipse.<\/p>\n
Again, I do not guarantee that the program will create error-free symbol hierarchy At best, use this code as a reference and modify it to suit your requirements. It is just one of the ways to create symbol heirachy and there could be better ways.<\/p>\n
-Ram Kulkarni<\/p>\n","protected":false},"excerpt":{"rendered":"
Last year I had blogged “Understanding AST created by Mozilla Rhino parser“, where I exaplained how to traverse AST to get all functions and variables. Since then I have received some comments, public and private, to provide sample code. So here is an example of how to parse JavaScript code and get all functions and … <\/p>\n
Continue reading “Parsing JavaScript code using Mozilla Rhino”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"Parsing JavaScript code using #Mozilla #Rhino http:\/\/wp.me\/p2g9O8-jw","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[20,76,1],"tags":[5,67,101],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2g9O8-jw","jetpack-related-posts":[],"_links":{"self":[{"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/posts\/1210"}],"collection":[{"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/comments?post=1210"}],"version-history":[{"count":0,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/posts\/1210\/revisions"}],"wp:attachment":[{"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/media?parent=1210"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/categories?post=1210"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/ramkulkarni.com\/blog\/wp-json\/wp\/v2\/tags?post=1210"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}