Understanding AST created by Mozilla Rhino parser

private AstRoot parse(String src, int startLineNum)
	throws IOException {

	CompilerEnvirons env = new CompilerEnvirons();
	env.setRecoverFromErrors(true);
	env.setGenerateDebugInfo(true);
	env.setRecordingComments(true);

	StringReader strReader = new StringReader(src);

	IRFactory factory = new IRFactory(env);
	return factory.parse(strReader, null, startLineNum);

}

root.visit(new NodeVisitor() {

	@Override
	public boolean visit(AstNode node) {
		int nodeType  = node.getType();
		//TODO: process the node based on node type
		return true; //process children
	}
});

function test()
{
	var k = 10;
}

i = 100;

15 Replies to “Understanding AST created by Mozilla Rhino parser”

1. Hello. Nice article. I am currently struggling with Rhino AST. You state that it is quite well documented, I have not found much in the way of documentation apart from the JavaDocs that have some gaps. Do you have a link to some useful documentation? Thanks again for the helpful article.

   Reply

   1. When I said that parsing with Rhino was well documented, I was talking about documentation for using parser APIs, not how AST was structured. However I can’t seem to find where I saw the documentation about using Parser APIs now. I am sure I saw it before. I might have also looked at test cases under testsrc/org folder, when you download Rhino.

      Reply

2. HI, thanks for the clear example. I was wandering if you know a way to change the function body. I currently want to wrap a function’s body with a “with(some_object){ }”

   Reply

   1. I haven’t done something like this before, but one possible solution could be that after you parse JS code, get all FunctionNode. You can get the start offset of each function by calling getAbsolutePosition method. The end offset could be startOffset + node.getText().length(). Once you know start and end positions, then insert wrapper code at those positions in the source file.

      Reply

3. I am new to Rhino parser. I need to find out how to fetch words present in given script. I have tried doing this but I could retrieve only names of variables and functions. But I need to extract every single word( which could become a bit difficult in obfuscated script). Here is my approach to find variable and function names in script

   import java.io.FileReader;
   import java.io.IOException;
   import java.io.Reader;

   import org.mozilla.javascript.Parser;
   import org.mozilla.javascript.ast.AstNode;
   import org.mozilla.javascript.ast.FunctionCall;
   import org.mozilla.javascript.ast.NodeVisitor;

   public class JavascriptParser {

   public static void main(String[] args) throws IOException {
   class Printer implements NodeVisitor {

   public boolean visit(AstNode node) {

   /* Here I am using Name instead of FunctionCall because it gives errors for a few functions present in script */

   if (node instanceof Name) {

   System.out.println(name.getString());
   /* this prints name of variable as well as functions present in script*/
   }
   return true;
   }
   }

   String file = “/dss2.js”;
   Reader reader = new FileReader(file);
   try {
   AstNode node = new Parser().parse(reader, file, 1);
   node.visit(new Printer());
   } finally {
   reader.close();
   }
   }

   Reply

   1. What do you mean by capturing every single word in JS script? With the script you posted, you will be able to capture all functions and variables. Can you give example of what you are not able to capture with your code?

      Reply

      1. Basically I wanted to find the ratio of keywords to word and probability of shell code being present in JS script (part of implementation of research paper) to check if script is vulnerable. So considering a dummy script like this :

         var emptyFun = function () {};

         var methods = [‘assert’, ‘clear’, ‘count’, ‘debug’, ‘dir’, ‘dirxml’, ‘error’,
         ‘exception’, ‘group’, ‘groupCollapsed’, ‘groupEnd’, ‘info’, ‘log’,
         ‘markTimeline’, ‘profile’, ‘profileEnd’, ‘table’, ‘time’, ‘timeEnd’,
         ‘timeStamp’, ‘trace’, ‘warn’];

         var length = methods.length;

         var console = (window.console = window.console || {});

         while (length1–) {
         method = methods[length];
         if (!console[method1]) {
         console[method] = emptyFun;
         }
         }

         In this case, from the previous code I will be able to extract [ emptyfun, methods, length, length1, console, window, method ] but I will not be able to extract words like [ function, assert, clear, etc]. Even though I have the script I cant separate words based on when a space character occurs (will not work in case of obfuscated script).

         Reply

         1. Just processing Name is not going to give you all words/keywords. You will have to handle each type of node separately. For example –

            public boolean visit(AstNode node) {
            	printNode(node);
            	return true;
            }
            
            void printNode (AstNode node)
            {
            	String nodeStr = "";
            	int type = node.getType();
            	switch (type)
            	{
            	case Token.NAME:
            		if (node instanceof Name)
            		{
            			Name name = (Name)node;
            			nodeStr =  name.getIdentifier();
            		}
            		break;
            	case Token.STRING:
            		if (node instanceof StringLiteral)
            			nodeStr = ((StringLiteral)node).getValue();
            		break;
            	case Token.WHILE:
            		nodeStr = "while";
            		break;
            	case Token.TRUE:
            		nodeStr = "true";
            		break;
            	case Token.FALSE:
            		nodeStr = "false";
            		break;
            	case Token.FUNCTION:
            		nodeStr = "function";
            		break;
            	default:
            		//find constant from Token.java and handle the case if you want.
            		System.out.println(String.valueOf(type));
            		return;
            	}
            	if (nodeStr.length() > 0)
            		System.out.println(nodeStr);
            }
            

            You will need to handle more types depending on what keywords you want to process. You can find more type constants in Token.java. If you have downloaded Rhino, you will find this class in rhino1_7R4/src/org/mozilla/javascript/Token.java

            Reply

            1. Thanks a lot. It worked. 🙂

4. I want to use rhino to get the AST of a piece of code and make some transformations on this tree. Then I try to generate the corresponding JavaScript code by the transformed AST, is there any API provided by rhino?

   Reply

   1. Not sure. I was only interested in parsing JS code, not regenerating it from AST – so did not look for such API. But if I find any such API, I will post it here

      Reply

5. thank you sir…your method printnode() in above comment is quite useful for me.
   but i have some problem..while running code ,if my javascript has some stmt like var a=10;
   then switch goes to default case and print final int value like 145. i want to print var name.
   and other problem is that if my script contains function code then various parse exception occurs.
   so please guide me.

   Reply

6. if stmt is like var a=eval(c+d);
   then please give me hint for how to reach eval() while parsing script.

   Reply

   1. The code snippet I posted in one of the comment for just an example of how to process nodes. I did not handle all the cases. To evaluate function calls, like eval, you can either check parent of node in Token.NAME case (in case of function call the parent would be of type FunctionCall) or handle a new case Token.CALL. Use debugger to see details of each node. Also refer to source of Token class of Mozilla to know what node type constants mean.

      Reply

7. Hello Ram,

   Your article is very helpful and thank you for writing such article.

   Where can I get all types of AST nodes? I have searched but no luck.

   Could you please help me how can I check the if below regular expression is present in my JS file after I have parsed it using AST?

   JS: steal(function($) {
   init: function(){
   this.validate(“irefNo”, function () {
   var test=””;
   test = this.iReferenceNumber.match(/[$+,:=?@#|!`^()~[]><\%*_=+]/g);
   return "Invalid characters' "+test+" 'found in field";
   }
   }
   }

   Reply