您的问题可以(至少)两种方式解释:
- 将大型语法中的规则分离为单独的语法;
- 解析“主要”语言中的一种单独的语言(岛屿语法)。
我认为这是第一个,在这种情况下您可以导入语法。
选项 1 的演示:
文件:L.g
lexer grammar L;
Digit
: '0'..'9'
;
文件:Sub.g
parser grammar Sub;
number
: Digit+
;
文件:根.g
grammar Root;
import Sub;
parse
: number EOF {System.out.println("Parsed: " + $number.text);}
;
文件:Main.java
import org.antlr.runtime.*;
public class Main {
public static void main(String[] args) throws Exception {
L lexer = new L(new ANTLRStringStream("42"));
CommonTokenStream tokens = new CommonTokenStream(lexer);
RootParser parser = new RootParser(tokens);
parser.parse();
}
}
运行演示:
bart@hades:~/Programming/ANTLR/Demos/Composite$ java -cp antlr-3.3.jar org.antlr.Tool L.g
bart@hades:~/Programming/ANTLR/Demos/Composite$ java -cp antlr-3.3.jar org.antlr.Tool Root.g
bart@hades:~/Programming/ANTLR/Demos/Composite$ javac -cp antlr-3.3.jar *.java
bart@hades:~/Programming/ANTLR/Demos/Composite$ java -cp .:antlr-3.3.jar Main
这将打印:
Parsed: 42
到控制台。
更多信息,请参阅:http://www.antlr.org/wiki/display/ANTLR3/Composite+Grammars http://www.antlr.org/wiki/display/ANTLR3/Composite+Grammars
选项 2 的演示:
语言中的语言的一个很好的例子是正则表达式。您拥有带有元字符的“正常”正则表达式语言,但其中还有另一种:描述字符集(或字符类)的语言。
而不是考虑字符集的元字符(范围-
, 否定^
等)在正则表达式语法中,您可以简单地将字符集视为由[
然后一切直到并包括]
(可能与\]
在它!)在你的正则表达式语法中。当你偶然发现一个CharSet
标记在您的解析器规则之一中,您可以调用 CharSet 解析器。
文件:正则表达式.g
grammar Regex;
options {
output=AST;
}
tokens {
REGEX;
ATOM;
CHARSET;
INT;
GROUP;
CONTENTS;
}
@members {
public static CommonTree ast(String source) throws RecognitionException {
RegexLexer lexer = new RegexLexer(new ANTLRStringStream(source));
RegexParser parser = new RegexParser(new CommonTokenStream(lexer));
return (CommonTree)parser.parse().getTree();
}
}
parse
: atom+ EOF -> ^(REGEX atom+)
;
atom
: group quantifier? -> ^(ATOM group quantifier?)
| EscapeSeq quantifier? -> ^(ATOM EscapeSeq quantifier?)
| Other quantifier? -> ^(ATOM Other quantifier?)
| CharSet quantifier? -> ^(CHARSET {CharSetParser.ast($CharSet.text)} quantifier?)
;
group
: '(' atom+ ')' -> ^(GROUP atom+)
;
quantifier
: '+'
| '*'
;
CharSet
: '[' (('\\' .) | ~('\\' | ']'))+ ']'
;
EscapeSeq
: '\\' .
;
Other
: ~('\\' | '(' | ')' | '[' | ']' | '+' | '*')
;
文件:字符集.g
grammar CharSet;
options {
output=AST;
}
tokens {
NORMAL_CHAR_SET;
NEGATED_CHAR_SET;
RANGE;
}
@members {
public static CommonTree ast(String source) throws RecognitionException {
CharSetLexer lexer = new CharSetLexer(new ANTLRStringStream(source));
CharSetParser parser = new CharSetParser(new CommonTokenStream(lexer));
return (CommonTree)parser.parse().getTree();
}
}
parse
: OSqBr ( normal -> ^(NORMAL_CHAR_SET normal)
| negated -> ^(NEGATED_CHAR_SET negated)
)
CSqBr
;
normal
: (EscapeSeq | Hyphen | Other) atom* Hyphen?
;
negated
: Caret normal -> normal
;
atom
: EscapeSeq
| Caret
| Other
| range
;
range
: from=Other Hyphen to=Other -> ^(RANGE $from $to)
;
OSqBr
: '['
;
CSqBr
: ']'
;
EscapeSeq
: '\\' .
;
Caret
: '^'
;
Hyphen
: '-'
;
Other
: ~('-' | '\\' | '[' | ']')
;
文件:Main.java
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;
public class Main {
public static void main(String[] args) throws Exception {
CommonTree tree = RegexParser.ast("((xyz)*[^\\da-f])foo");
DOTTreeGenerator gen = new DOTTreeGenerator();
StringTemplate st = gen.toDOT(tree);
System.out.println(st);
}
}
如果你运行主类,你会看到点输出 http://www.graphviz.org/Documentation.php对于正则表达式((xyz)*[^\\da-f])foo
这是下面的树:
魔法就在里面Regex.g
语法中的atom
规则,我通过调用静态在重写规则中插入树节点ast
方法从CharSetParser
class:
CharSet ... -> ^(... {CharSetParser.ast($CharSet.text)} ...)
请注意,在此类重写规则中,必须有not是一个分号!所以,这是错误的:{CharSetParser.ast($CharSet.text);}
.
EDIT
以下是为两种语法创建树遍历器的方法:
文件:RegexWalker.g
tree grammar RegexWalker;
options {
tokenVocab=Regex;
ASTLabelType=CommonTree;
}
walk
: ^(REGEX atom+) {System.out.println("REGEX: " + $start.toStringTree());}
;
atom
: ^(ATOM group quantifier?)
| ^(ATOM EscapeSeq quantifier?)
| ^(ATOM Other quantifier?)
| ^(CHARSET t=. quantifier?) {CharSetWalker.walk($t);}
;
group
: ^(GROUP atom+)
;
quantifier
: '+'
| '*'
;
文件:CharSetWalker.g
tree grammar CharSetWalker;
options {
tokenVocab=CharSet;
ASTLabelType=CommonTree;
}
@members {
public static void walk(CommonTree tree) {
try {
CommonTreeNodeStream nodes = new CommonTreeNodeStream(tree);
CharSetWalker walker = new CharSetWalker(nodes);
walker.walk();
} catch(Exception e) {
e.printStackTrace();
}
}
}
walk
: ^(NORMAL_CHAR_SET normal) {System.out.println("NORMAL_CHAR_SET: " + $start.toStringTree());}
| ^(NEGATED_CHAR_SET normal) {System.out.println("NEGATED_CHAR_SET: " + $start.toStringTree());}
;
normal
: (EscapeSeq | Hyphen | Other) atom* Hyphen?
;
atom
: EscapeSeq
| Caret
| Other
| range
;
range
: ^(RANGE Other Other)
;
主程序.java
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;
public class Main {
public static void main(String[] args) throws Exception {
CommonTree tree = RegexParser.ast("((xyz)*[^\\da-f])foo");
CommonTreeNodeStream nodes = new CommonTreeNodeStream(tree);
RegexWalker walker = new RegexWalker(nodes);
walker.walk();
}
}
要运行演示,请执行以下操作:
java -cp antlr-3.3.jar org.antlr.Tool CharSet.g
java -cp antlr-3.3.jar org.antlr.Tool Regex.g
java -cp antlr-3.3.jar org.antlr.Tool CharSetWalker.g
java -cp antlr-3.3.jar org.antlr.Tool RegexWalker.g
javac -cp antlr-3.3.jar *.java
java -cp .:antlr-3.3.jar Main
这将打印:
NEGATED_CHAR_SET: (NEGATED_CHAR_SET \d (RANGE a f))
REGEX: (REGEX (ATOM (GROUP (ATOM (GROUP (ATOM x) (ATOM y) (ATOM z)) *) (CHARSET (NEGATED_CHAR_SET \d (RANGE a f))))) (ATOM f) (ATOM o) (ATOM o))